Vals AI ran Inception's Mercury 2.5 across its benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → suite and reported the model trades away accuracy for speed. Vals AI's own model page shows Mercury 2.5 scoring about 12.95% on its overall Vals Index, ranking 65th of 66 models tested, and 12.77% on its Tax AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → Bench, ranking 61st of 61. Vals AI's tweet describes the model only as extremely fast without stating specific timing figures in the post itself, and the chart image accompanying the tweet was not reviewed here, so the exact speed advantage over other models is not established beyond that general description.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
