Artificial Analysis

Meta has released Muse Spark 1.2. It's their third release in four months and scores 54 on the Artificial Analysis Intelligence Index, significantly improving agentic knowledge work capabilities over prior releases and putting Meta next to SpaceXAI in a tie for third place https://t.co/460hp9zIxB

Alibaba's Qwen3.8 Max scores 56 on the Artificial Analysis Intelligence Index at $1.14 per task, but open weights leader Kimi K3 remains 1 point ahead at 25% lower cost per task ($0.86) @Alibaba_Qwen has released Qwen3.8 Max, which Alibaba states is a 2.4T total parameter MoE https://t.co/Wix4sXPd09

Google has released Gemini 3.7 Flash, improving 4 points over Gemini 3.6 Flash and reaching the Intelligence vs. Time per Task Pareto frontier @GoogleDeepMind has released its third new Gemini Flash model in three months. Gemini 3.7 Flash (high) scores 56 on the Artificial Analysis Intelligence Index, just behind GPT-5.6 Terra (max, 57) and Muse Spark 1.2 (xhigh, 57) We benchmarked Gemini 3.7 Flash across all three reasoning levels (high, medium, low) ahead of release. With high reasoning, Gem

We have updated the Artificial Analysis Intelligence Index to v4.1.1 - this patch release upgrades our grader models, and brings the latest 𝜏³-Banking version to Artificial Analysis To keep the Artificial Analysis Intelligence Index the most useful synthesis metric for https://t.co/GK7KKRRTcs

DeepSeek V4 Pro 0813 scores 53 on the Artificial Analysis Intelligence Index, 8 points above April's DeepSeek V4 Pro - but with a 3.6x price increase and only 1 point above DeepSeek V4 Flash 0731 @deepseek_ai has released DeepSeek V4 Pro 0813, its new flagship model, along with updated pricing. With the new pricing, it still sits on our Pareto frontier for Intelligence vs. Cost, but by a smaller margin than previous DeepSeek releases. Under the new pricing, DeepSeek’s first-party API will char

Grok 4.6 made large gains on AA-Briefcase, our agentic knowledge work benchmark and cost substantially less than other leading models AA-Briefcase tests models on long-horizon agentic knowledge work tasks. The test set is private to prevent contamination. Grok 4.6 is neck and neck with Claude Fable 5, with overlapping confidence intervals. The model is also substantially cheaper than other leading models on the benchmark at a Cost per Task of $4.42 compared to Claude Fable 5's $22.30, Claude

Meta returns to open weights: Muse Glimmer, its first open-weights release since Llama 4, scores 35 on the Artificial Analysis Intelligence Index. It is a 30B-parameter model, and the first from Meta to be released under Apache 2.0 Muse Glimmer (high) arrives 16 months after Llama 4, scoring 21 points above Llama 4 Maverick (14), Meta's last open weights release. It sits alongside Kimi K2.5 (Reasoning, 36) and just behind Qwen3.6 27B (Reasoning, 38) and Ling 3.0 Flash (38), and creates a two-ti

NVIDIA has just released the first Nemotron 3.5 model: Nemotron 3.5 Lightning, a highly efficient small open weights model with performance similar to gpt-oss-120b at around a quarter of the total parameters Nemotron 3.5 Lightning is the successor to @nvidia Nemotron 3 Nano 30B A3B, with 31.6B total and 3.6B active parameters. It retains the same hybrid Mamba-Transformer architecture and small size from Nemotron 3 Nano, but makes substantial gains in intelligence and agentic performance. Key

Only two labs occupy the Time per Task Pareto frontier: all frontier models under two minutes are from OpenAI, and four of the six above are from Anthropic We show the tradeoff between the Artificial Analysis Intelligence Index and how long it takes models to complete a https://t.co/yXjHYDCzrI

Korean AI lab Upstage has released Solar Pro 4, scoring 42 on the Artificial Analysis Intelligence Index, a significant increase from Solar Pro 3’s 14 Solar Pro 4 is @upstageai's new proprietary flagship reasoning model, replacing Solar Pro 3 from April 2026. At 42 on the Intelligence Index it sits alongside Inkling (xhigh, 42) and just behind MiMo-V2.5-Pro (43), and shows a 27-point increase over Solar Pro 3. Pricing increases to $0.30/$1.20/$0.06 per 1M input/output/cache hit tokens from Sola

Ant Group has just released Ling 3.0 Flash, a 124B open weights model that scores 38 on the Artificial Analysis Intelligence Index. Ling 3.0 demonstrates a marked improvement over the previous generation and sits on the Pareto frontier for Intelligence versus Total Parameters https://t.co/gOEOE5o8Rv

Announcing AA-AnalystAgent, our new agentic benchmark for quantitative analysis on real-world spreadsheets & documents. Claude Opus 5 leads at 54%, followed by GPT-5.5 at 50% and Claude Fable 5 at 49% In real analyst roles, professional judgment and expertise are as important raw quantitative capabilities. AA-AnalystAgent tests this and requires models to interpret sources, decide which exceptions and caveats apply, and settle on a methodology to successfully complete a task. Because handing

Muse Spark 1.2 places Meta on the Cost per Task Pareto frontier, scoring 6 points below Claude Opus 5 at ~1/6th of the cost At Meta's $1.25/$4.25 per 1M token pricing, Muse Spark 1.2 (xhigh) sits on the Pareto frontier of Intelligence Index vs Cost per Task. It delivers https://t.co/s2JS6ZTZkO

We have updated the Artificial Analysis Text to Image Arena to expand the range of use cases and capabilities we test for, from UI/UX Design to Live-Action Film, and from Physics to Text Rendering. Our updated evaluation measures not just which model is best overall, but which https://t.co/2xpdLdnDHp

Ant Group has released Ling 3.0 Tiny, which scores 25 on the Artificial Analysis Intelligence Index. With just 7.9B total and 1.3B active parameters, it sits on the Pareto frontier for Intelligence vs. Active Parameters @AntLingAGI has released Ling 3.0 Tiny, an open weights reasoning model with a 262K-token context window. It scores 25 on the Artificial Analysis Intelligence Index, comparable to gpt-oss-120b (high, 24) with 15x fewer total parameters and 4x fewer active parameters. However, th

Muse Spark 1.2 is among the most cost-efficient models at its intelligence level: $0.40 per Intelligence Index task at Meta's $1.25/$4.25 per 1M token pricing. That compares to $0.51 for GPT-5.6 Terra (max), $0.86 for Kimi K3 (max), and $1.18 for GPT-5.5 (xhigh) - with Grok 4.5 https://t.co/FcCMeyt37X

The 3-point gain over Muse Spark 1.1 on the Artificial Analysis Intelligence Index is concentrated in agentic evaluations: GDPval-AA v2 +260 Elo (1371 to 1631), Terminal-Bench 2.1 +2 points (78% to 80%), and Tau3-Bench Banking +2 points (25% to 27%). The minor regressions are https://t.co/7MzPZxu8ZB

Qwen3.8 Max scores 1739 Elo on GDPval-AA, ahead of Kimi K3 (1685), effectively tied with Claude Fable 5 (1743) and GPT-5.6 Sol (max, 1730), and behind only Claude Opus 5 (max, 1852). The 468 Elo gain is in part driven by the new model taking more turns per task (64 vs. 14 for https://t.co/23aQXrTnqS

Qwen3.8 Max costs $1.14 per Intelligence Index task, more than double Qwen3.7 Max ($0.53) and ~1.3x open weights leader Kimi K3 (max, $0.86). The increase is driven by more agentic turns rather than pricing: per-token prices fell from Qwen3.7 Max ($2.00/$6.00 vs $2.50/$7.50 per https://t.co/TmOR26GzES

AA-Omniscience regresses 10 points from Qwen3.7 Max (+14 to +4), reversing its predecessor's abstention gains. Accuracy is effectively flat at ~31% while the hallucination rate rises from 23% to 40%, meaning Qwen3.8 Max attempts more questions it cannot answer rather than https://t.co/UClHxJwHKd

Muse Spark 1.2 ranks #5 on GDPval-AA v2, our benchmark of agentic real-world knowledge work, at 1631 Elo - behind e.g. Claude Opus 5 (max, 1852), GPT-5.6 Sol (max, 1730), and Kimi K3 (1685), and ahead of e.g. Claude Opus 4.8 (max, 1588). Muse Spark 1.1 scored 1371 at its launch https://t.co/nbaNwsYvKU

Muse Spark 1.2 continues Meta's AA-Omniscience pattern: the score rose from 18 to 22, driven by abstention for the second consecutive release. The hallucination rate fell 10 points (38% to 28%) as the attempt rate dropped to 67%, while accuracy slipped from 41% to 38%. Heavy https://t.co/IXjZRsKpyT

Muse Glimmer's gaps against its class concentrate in agentic evaluations: 953 Elo on GDPval-AA v2 against 1141 for Qwen3.6 27B (Reasoning), 1141 for Gemini 3.5 Flash-Lite, and 1004 for Kimi K2.5 (Reasoning), with Terminal-Bench v2.1 (52%) also behind Qwen3.6 27B (61%). The hallucination gap follows: 82% on AA-Omniscience against 49% for Qwen3.6 27B and 34% for Flash-Lite (lower is better). Its size-twin Gemma 4 31B (Reasoning) performs worse than Muse Glimmer on all of these measures. The except

Ling 3.0 Flash scores -18 on AA-Omniscience, a 48 point improvement from Ling 2.6 Flash’s -66. This is driven primarily by significant decrease in hallucination rate from 97% to 44%. https://t.co/O0TQzFIw03

Failure analysis: We classified 1,567 failing AA-AnalystAgent attempts across ten leading models into seven failure modes, tagging each attempt with every mode it exhibited. The most widespread is anchoring on a wrong early hypothesis, present in 57% of failures. Models commit to a source or interpretation early and defend it for the rest of the trajectory. The sharpest contrasts are in how models treat sources. Gemini 3.1 Pro Preview takes sources at their word but struggles with execution. It