A small model's low advertised tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → price can hide expensive long-running work. Artificial Analysis' Claude Haiku 5.5 launch evaluation reported heavy output use at maximum effort and a higher pricing tier for long prompts. Its provisional cost figures did not yet include that tier change.
The report therefore gives stronger evidence about the tested capabilities than about a fully comparable final bill. For Anthropic's model, both effort level and prompt length affect the economics. The meaningful comparison is the cost of completing a representative workload at adequate quality, rather than assuming that a model sold as the cheaper sibling will always be cheaper for the same job.
Key takeaways · AI-distilled
Artificial Analysis says Haiku 5.5's score of 43 is up 26 points a year after the last Haiku, slightly ahead of GLM-5.3 Flash (42), Gemini 3.8 Flash (41) and GPT-6 Luna (38), comparable to Kimi K3 (44), and 13 behind Claude Sonnet 5.5 (max).
Below 100k tokens Haiku 5.5 costs the same as GPT-6 Luna and a tenth of the previous Haiku, but it is token-hungry: about 162k output tokens per task at max effort, roughly 3x Luna, with xhigh to max adding 2 points for about 1.8x the tokens.
It is the first Haiku with effort settings and adaptive thinking, and its context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → grows to 1M tokens from 200k; Artificial Analysis notes its site's provisional cost figures do not yet include the step-up price above 100k tokens.
Agentic results are mixed: 1578 Elo on AA-Briefcase, within the confidence intervals of GPT-6 Astra and Fable 5.1 (high), and 33% on Terminal-Bench 4.0 versus 0% for Haiku 4.5, but 35% on AutomationBench-AA, which AA blames on a pre-release over-refusal issue.
Its AA-Omniscience accuracy is 36% against 55% for Gemini 3.8 Flash and 44% for GPT-6 Luna, partly because it more often admits not knowing: its hallucinationWhen a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.Full definition → rate is 40% versus 55% and 77%.
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
