Artificial Analysis: Haiku 5.5 leads small models but burns tokens
- Source
- x.com
- Date
Anthropic has released Claude Haiku 5.5, scoring 43 on the Artificial Analysis Intelligence Index - up 26 points one year after the last Haiku release Haiku 5.5 is the first Haiku model with Anthropic’s effort settings and adaptive thinking, and Anthropic has introduced tiered pricing. Haiku 5.5 is cheaper than its predecessor - it costs $0.10/$0.50 per 1M input/output tokens for prompts up to 100k tokens (the same as GPT-6 Luna and 10% of the previous Haiku model). However, this pricing rises 5x to $0.50/$2.50 above 100k. The site does not yet reflect tiered pricing, so provisional cost figures for Haiku 5.5 do not include the step up cost. We are working on support and will follow up with Cost per Task coverage soon. Key takeaways: ➤ Leading small-class model performance: At max effort Haiku 5.5 sits slightly ahead of models such as GLM-5.3 Flash (42), Gemini 3.8 Flash (41) and GPT-6 Luna (38). Its score is comparable to Kimi K3 (44), a 2.8T parameter open weights model, and trails Claude Sonnet 5.5 (max, 56) by 13 points ➤ Heavy token use compared to GPT-6 Luna: Haiku 5.5 (max) uses ~162k output tokens per Intelligence Index task, ~3x GPT-6 Luna (max, ~50k). Moving from…
Context
Artificial Analysis, a benchmarking group, reports that Anthropic's Claude Haiku 5.5 leads the small-model class on its Intelligence Index but uses far more output to get there. In its October 7 thread, it gives the model 43 at max effort, slightly ahead of GLM-5.3 Flash at 42, Gemini 3.8 Flash at 41 and GPT-6 Luna at 38, and 26 points above the last Haiku.
The catch is use, since tokens are the units of text that models read and are billed for. At max effort Haiku 5.5 uses about 162,000 output tokens per task, roughly three times Luna's 50,000 at max. At equal scores a gap remains: Haiku 5.5 at high effort scores 38 with about 55,000 tokens, against 38 with about 50,000 for Luna at max. Haiku's price is $0.10 per million input tokens and $0.50 per million output tokens up to 100,000 tokens, the same as Luna, and five times that above. Artificial Analysis says its cost figures do not yet include the higher tier and promises cost-per-task coverage, so the cost of finishing a task is not yet reported.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Checking sign-in…
Loading comments…