Kimi K3.1 Pro lands 4th on Artificial Analysis with big agentic gains

Kimi K3.1 Pro narrows the gap between open and proprietary models to 3 index points once weights ship. It tops Terminal-Bench 4.0 at $1.98 per task, with token prices 33% lower than K3.

• Kimi K3.1 Pro scores 55 on the Artificial Analysis Intelligence Index, 11 points over Kimi K3 and behind only Claude Opus 5.5, Claude Sonnet 5.5 and Muse Wonder Full. AA says published weights would cut the open-versus-proprietary gap from 12 points to 3.
• The biggest gains are agentic: Terminal-Bench 4.0 jumps from 13% to 67%, the highest of any model per AA; GDPval-AA rises 312 Elo to 1,846; and AutomationBench-AA climbs from 58% to 73%. Knowledge and science evals moved only a few points.
• Prices fell 33% to $2/$10/$0.20 per 1M input/output/cache-hit tokens, but the model used 14% more output tokens per task, so AA puts cost per task at $1.98, roughly equal to Kimi K3. Weights were not yet published at launch.

• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

• postDeepSeek V4.1 Flash 1017 tops open models on cyber defense index
• postMuse Spark Open inference benchmarks: ~150M tokens/s per MW on B300
• postKimi K3.1 Pro leads open models on agentic cyber defense benchmark
• postOn MLCR-AA and HealthBench Professional, Anthropic models give the most…

• articleKimi K3 1 Pro Launch BenchmarksArtificial Analysis articles
• postKimi K3.1 Pro leads open models on agentic cyber defense benchmarkArtificial Analysis
• articleIntroducing Kimi K3.1 Prosimonwillison.net

Checking sign-in…

Loading comments…