Intelligence Index v5 moves benchmarks toward private, agentic tests

Version 5 of the Artificial Analysis Intelligence Index, a composite score built from several benchmarks, now rests mostly on tests whose questions or answers are kept out of public view. According to the v5 announcement, 60% of the Index now comes from evaluations with private test sets or held-out answers. The Index is split evenly across four areas at 25% each: knowledge work, coding, science and general knowledge. Two new benchmarks carry much of the change. FrontierCode 2.0, a private benchmark from Cognition, has 132 tasks in real open-source repositories and grades an agent on whether a maintainer would merge its change; a change that fails any blocking check scores zero, and the tasks stay private to avoid contamination. Terminal-Bench-Science has 70 scientific research tasks carried out in a terminal, and a task passes only if all its tests pass. Each carries 15% of the Index. HLE-Diamond replaces Humanity's Last Exam, and SciCode, GDPval-AA v2.1 and AA-LCR v1.1 were dropped.

The harder, more agentic mix separates models more. Claude Opus 5.5 at max effort leads at 58, followed by GPT-6 Astra at 57 and GPT-6.1 Sol at 55, and Anthropic and OpenAI hold the top five places. No model leads all nine evaluations. The gap between the best open-weights model and the leader grows from 3 points in v4.3 to 12. That best open model, Kimi K3.1 Pro at 46, has not had its weights released yet. Among models whose weights are out, DeepSeek V4.1 Flash 1017 and GLM-5.3 lead at 32, below Anthropic's small Claude Haiku 5.5 at 36. The gap is widest on HLE-Diamond and Terminal-Bench-Science.

The new tasks also take much more work to complete, which makes cost and effort settings part of the result. Every model measured produced more output tokens per Index task than under v4.3, and the median rose from 35,000 to 59,000. The four agentic evaluations make up 55% of the Index weight but a median of about 90% of output tokens. Claude Opus 5.5 scores 35 at low effort and 58 at max effort, and the max-effort run costs ten times as much per task, $10.04 against $0.98. GPT-6.1 Sol at max effort comes within 4 points of Opus 5.5 for about an eighth of the cost per task. A v5 score is therefore a statement about a model at a particular effort setting, and the price of reaching it varies widely.

Because the benchmark mix changed, v5 scores cannot be read against earlier Index versions. Artificial Analysis launched the Progress Index alongside v5 for comparisons over time and keeps the Intelligence Index for choosing among current models. The v5 rankings have already moved: a day after launch, Artificial Analysis reported that Gemini 4 Carbon had taken first place.