Independent numbers show Kimi K3.1 Pro near the frontier on agentic evals, including the top Terminal-Bench 4.0 score, at about $1.98 per task. Once its weights are published, open models will trail proprietary ones by only 3 points on this index.
Key takeaways · AI-distilled
Artificial Analysis ranks Kimi K3.1 Pro 4th on its Intelligence Index at 55, up 11 points from Kimi K3, behind Claude Opus 5.5 (58), Claude Sonnet 5.5 (56) and Muse Wonder Full (56).
The gains are concentrated in agentic evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →: Terminal-Bench 4.0 rises from 13% to 67%, GDPval-AA gains 312 Elo to 1,846, and AutomationBench-AA, which tests agentic SaaS workflows, climbs from 58% to 73%.
Other evals move less: Humanity's Last Exam goes from 47% to 52% and SciCode from 59% to 61%. On AA-Omniscience the hallucinationWhen a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.Full definition → rate falls from 53% to 44% while accuracy rises from 48% to 51%.
Cost per task stays about flat at $1.98: tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → prices are 33% lower than Kimi K3 ($2/$10/$0.20 per 1M input/output/cache hit), but the model uses 14% more output tokens per task (55K vs 48K).
The model has 2.8T total and 104B active parameters, a 1M-token context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →, and text plus image input.
Terms in this piece · Glossary
hallucination — When a model states something false with full confidence — inventing facts, citations, or APIs that don't exist.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
