Claude Opus Vs Sonnet Vs Haiku
- Source
- openrouter.ai
- Author
- OpenRouter editorial sitemap
- Date
Context
OpenRouter, a service that gives access to many AI models through one API, tested Anthropic's three Claude 5.5 tiers and found that on short, well-specified coding tasks the expensive model bought nothing. In its comparison, Opus 5.5, Sonnet 5.5 and Haiku 5.5 each passed all 25 attempts at 11 Python tasks with hidden tests, and all three fixed every bug in a small invoicing library in three Claude Code runs each. With no difference in results, cost separated them: Haiku was 19 to 30 times cheaper than Opus per task, and Sonnet cost half to two-thirds as much as Opus. List prices per million input and output are $4 and $20 for Opus, $2 and $10 for Sonnet, and $0.10 and $0.50 for Haiku.
OpenRouter's own runs pointed the same way. Sonnet scored above Opus on GPQA Diamond, 198 graduate-level science questions (92.4% against 90.4%), and on τ-bench airline, a simulated multi-turn customer-service task (78.7% against 75.5%), at less than half the cost per question or task. Opus leads by 1.6 points on the composite index from Artificial Analysis, an independent benchmarking firm. Haiku scored 76.0% on τ-bench but trails Sonnet by 5.8 points on GPQA and 12.6 on that index. OpenRouter notes that with 100 to 200 tasks per model, gaps of a few points on τ-bench may be noise.
Its recommendation is Sonnet as the default, Haiku for high-volume and work, and Opus for long, ambiguous tasks where a mistake costs more than the tokens. That carve-out covers the long-running agentic work Anthropic positions Opus for, and OpenRouter says its short tasks cannot rule out an Opus advantage there. The question matters because many people run Opus without choosing it: it is Claude Code's default on most plans, and in the week of October 12 it carried about two and a half times Sonnet's token volume on OpenRouter.
The tests also exposed a billing quirk. Claude Code's own cost field did not recognize OpenRouter's Haiku model name and reported $0.16 to $0.31 per session, against a computed median of about half a cent, so people running Claude Code through OpenRouter should check spend on OpenRouter's Activity page instead. The same bug-fix test anchors OpenRouter's companion breakdown of Claude Code session costs.
- On OpenRouter's own runs, Sonnet 5.5 beat Opus 5.5 on GPQA Diamond (92.4% vs 90.4%) and tau-bench airline (78.7% vs 75.5%) at under half the cost per task; Opus leads only by 1.6 points on the Artificial Analysis index.
- All three tiers passed all 25 attempts on 11 hidden-test Python tasks, including a regex engine without the re module and SQL three-valued NULL logic; on hard tasks Haiku used 3,202 reasoning tokens to Opus's 1,252 yet finished fastest.
- OpenRouter cautions its tasks were short and well specified, so they cannot rule out an Opus advantage on long-running agentic work, and with 100 to 200 tau-bench tasks per model, gaps of a few points may be noise.
- Claude Code 2.1.307 does not recognize OpenRouter's Haiku 5.5 slug and assumes a 200,000-token window; OpenRouter says adding [1m] to the slug restores the full 1-million-token , and it strips the suffix before routing.
- Opus 5.5 carried about 2.5 times Sonnet's OpenRouter token volume in the week of October 12 and is Claude Code's default on most plans, so users who never set a model are running the most expensive of the three tiers.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- subagent — A separate agent spawned by another to do one scoped piece of work in its own context, returning only the result.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…