Haiku 5.5 leads small models on benchmarks but spends far more tokens
- Source
- x.com
- Date
Anthropic has released Claude Haiku 5.5, scoring 43 on the Artificial Analysis Intelligence Index - up 26 points one year after the last Haiku release Haiku 5.5 is the first Haiku model with Anthropic’s effort settings and adaptive thinking, and Anthropic has introduced tiered pricing. Haiku 5.5 is cheaper than its predecessor - it costs $0.10/$0.50 per 1M input/output tokens for prompts up to 100k tokens (the same as GPT-6 Luna and 10% of the previous Haiku model). However, this pricing rises 5x to $0.50/$2.50 above 100k. The site does not yet reflect tiered pricing, so provisional cost figures for Haiku 5.5 do not include the step up cost. We are working on support and will follow up with Cost per Task coverage soon. Key takeaways: ➤ Leading small-class model performance: At max effort Haiku 5.5 sits slightly ahead of models such as GLM-5.3 Flash (42), Gemini 3.8 Flash (41) and GPT-6 Luna (38). Its score is comparable to Kimi K3 (44), a 2.8T parameter open weights model, and trails Claude Sonnet 5.5 (max, 56) by 13 points ➤ Heavy token use compared to GPT-6 Luna: Haiku 5.5 (max) uses ~162k output tokens per Intelligence Index task, ~3x GPT-6 Luna (max, ~50k). Moving from…

Context
Artificial Analysis, a benchmarking group, finds that Anthropic's Claude Haiku 5.5 is the strongest small on its Intelligence Index, but that its low price per does not settle whether it is cheap to use. In its launch-day thread, the group scores Haiku 5.5 at 43 at its highest effort setting, 26 points above the previous Haiku and slightly ahead of GLM-5.3 Flash at 42, Gemini 3.8 Flash at 41 and GPT-6 Luna at 38.
The price question comes from how much text the model produces. Tokens are the units of text a model reads and writes, and the bill is per token. For prompts up to 100,000 tokens, Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, which Artificial Analysis says matches GPT-6 Luna and is 10% of the previous Haiku's price. But at max effort Haiku 5.5 writes about 162,000 output tokens per index task, roughly three times Luna's 50,000. Haiku 5.5 is the first Haiku with effort settings, which let a developer trade more reasoning for more tokens, and the thread shows that trade gets expensive at the top: moving from the xhigh setting to max adds 2 points for about 1.8 times the tokens. Even when the two models are matched at a score of 38, Haiku 5.5 at high effort uses about 55,000 tokens per task against about 50,000 for Luna, and the group says the gap widens at lower settings.
Prompts longer than 100,000 tokens cost five times as much, and Artificial Analysis says its site does not yet reflect that step, so its provisional cost figures for Haiku 5.5 leave it out. The group has not yet published its cost-per-task measurements, so the thread does not say how much a finished task costs on Haiku 5.5 compared with Luna. It also flags one score as likely too low: on AutomationBench-AA Haiku 5.5 scored 35%, against 53 to 60% for its rivals, because a safety-refusal problem in pre-release testing made the model refuse too often, and the group plans to re-run it once Anthropic's fix lands. Anthropic's own launch post claims Haiku 5.5 costs around 75% less to run than Haiku 4.5 on average, a comparison against its predecessor rather than against Luna.
- Model — A computational system that uses learned patterns to produce an output from information supplied to it.
- Token — A piece of text, such as a word, part of a word or a punctuation mark, that a language model represents as numbers and handles as a unit.
- Terminal — A program that shows text and accepts typed text.
- Hallucination — Generated content that is false or unsupported, even when it sounds plausible.
Checking sign-in…
Loading comments…






