Routers Are a New AI Primitive

A model router sits in front of a pool of models and decides, for each request, which one should answer, weighing quality, cost and latency. Vals AI argues in this post that routers matter because the frontier is jagged: on its Vals Index, five different models hold the top spots across eight benchmarks, and none ranks in the top three on all of them. Its table shows how wide the gaps get. Claude Sonnet 5.5 and Opus 5.5 rank second and third overall but 35th and 33rd on Harvey's Legal Agent Benchmark, which Meta's Muse Spark 1.2 tops while ranking 33rd overall.

To size the opportunity, Vals computed an oracle router, one that picks the perfect model for every task in hindsight. On the Vals Index it would beat the best single model by 14 points while costing about 40% less. That is a ceiling, not a measurement of any product. Vals says existing academic router benchmarks cannot show how much of it real routers capture, because most score one choice on one public question, often from saturated datasets where every model is right and a router that always picks the cheapest model looks perfect. Agent work is harder: routing happens at every step of tasks that can run for hours, a cheap model's mistake can take several calls to recover from, and switching models discards the existing prompt cache, so a cheaper model does not automatically save money.

Vals proposes swapping each router in for the model inside its benchmark harnesses, consulting it on every model call, running it as shipped with its default pool, and comparing task outcomes, cost per task, cache usage and completion time against the best and the cheapest single model in that router's own pool, with the oracle as a reference point. Six of the Index's eight benchmarks are private, which reduces the risk that test tasks were seen in training. The post acknowledges that providers already publish their own evaluations, including OpenRouter's router benchmarks and Fireworks' live A/B tests that account for lost caching. It contains no router scores yet; Vals says results will appear on a public leaderboard soon.