Artificial Analysis
2 Tools · 276 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
Artificial Analysis's AA-Briefcase-Lite results show Grok 4.7 gaining sharply in Analytical Quality Elo (1698 to 1994) over Grok 4.6, trailing only Anthropic's models, while costing about half of Opus 5's per-task cost on market-analysis and deal-assessment work.
Artificial Analysis adds trusted-access models to its Cyber Index. OpenAI's Daybreak-only GPT-6 Sol leads, improving 32 points over the public model, and costs $1.77 per task against $11.67 for Grok 4.7.
Artificial Analysis introduces AA-PersonalityBench and reports sycophancy rankings. GPT-4o, Llama 4 Maverick and GPT-3.5 Turbo rank highest, Claude models all rank 55 or below, and Grok 4.7 ranks lowest. Methodology and the full article were not read, so the measure's validity is unchecked.
Artificial Analysis introduces AA-3D, which has agents build three.js scenes from written briefs in an offline sandbox, then ranks models by blind pairwise human votes. Anthropic and OpenAI models lead in a statistical tie; open-weights models trail by 200+ Elo.
Anthropic's Claude Fable 5 is the first public Mythos-class model, ranks first on GDPval-AA, and adds safeguards that can route flagged queries to Claude Opus 4.8 as a fallback.
Artificial Analysis finds Kimi K2.6 scores 54, behind only Anthropic, Google and OpenAI, with large agentic gains, lower hallucination, high token usage, and open weights on 1T total parameters.
Artificial Analysis evaluates Sarvam's 105B and 30B open-weights MoE models, scoring 18 and 12 on its index, with notable gaps on terminal coding and very high hallucination rates.
Artificial Analysis launches AA-Omniscience: 6,000 questions across 42 topics scoring +1 correct, -1 incorrect, 0 abstain. All but three models hallucinate more than they answer correctly on hard questions; 600 questions are open-sourced.
Artificial Analysis introduces AA-3D, a blind human-voted benchmark where agents with Blender, Rapier physics and a headless browser build three.js scenes from briefs. It ranks 21 models by Elo, shows a top-tier statistical tie, a wide open-weights gap, and a 160x spread in cost per scene.
Artificial Analysis revises its speech-to-text benchmark with AA-AgentTalk, a proprietary held-out set of voice-agent speech, cleaned VoxPopuli and Earnings22 transcripts, a better text normalizer, and new weighting.
Artificial Analysis benchmarks show GPT-6 Sol and Luna roughly halving cost per task versus GPT-5.6 while intelligence scores stay flat, with Sol improving and Luna regressing on the Coding Agent Index.
Artificial Analysis rates Claude Opus 5.5 at 58 on the Intelligence Index, the highest it has measured, with 59.6% on Terminal-Bench 4.0, $4/$20 pricing and a 95% cache discount.
Artificial Analysis ranks fal's FLUX 3 Max video model #2 on image-to-video and #4 on text-to-video, tied with Seedance 2.5 and MiniMax H3. It leads in Physics, costs $10.20/min versus $17.40 for FLUX 3, and has a 75% launch discount.
Cartesia's Sonic 3.6 takes the top of Artificial Analysis's controlled-voice arena at 1,117 Elo, ahead of Sonic 3.5, ElevenLabs v3 and Alibaba's Qwen-Audio-3.0-TTS-Plus. The full table pairs Elo with pricing that ranges from about $10 to $100 per million characters, including open-weights entries.
Artificial Analysis rebuilt its Image Editing Arena around seven editing actions and ten use cases with chained edit instructions. MAI-Image-2.6-Preview leads overall, GPT Image 2 wins precise local edits, Seedream 5.0 Pro leads identity preservation, and prices range from $20 to $211 per 1,000 imag
Artificial Analysis benchmarks Meta's Muse Spark Open, an 816B MoE with 42B active. It scores 39, leads open models on AA-Omniscience, uses about 40k tokens per task, and runs on four Blackwell Ultra GPUs or two Mac Studios. Agentic evals are its weak spot.
Artificial Analysis launches the Search Index comparing search API providers inside the Stirrup harness with GPT-5.6 Luna, scored on DeepSearchQA, BrowseComp and AA-Omniscience against a no-search baseline.
Artificial Analysis benchmarks Sonnet 5.5 in Claude Code across effort levels. Max scores 68 at 4.8x the output tokens and 3.4x the turns of xhigh, which scores 63 at 77% lower cost. Terminal-Bench 4.0 rises from 58.1% to 66.2%.
Artificial Analysis adds leaderboards for OpenAI's HealthBench and HealthBench Professional, reporting raw and length-adjusted scores, cost per task, and a Pareto frontier across GPT-6, Claude and DeepSeek models. Length penalties shift Claude rankings.
Artificial Analysis benchmarks Cohere's open-weights Command A+: 37 on its index, top non-hallucination rate, about 281 tokens per second, and weaker coding and scientific reasoning.
Artificial Analysis benchmarks three US open-weights releases from one week: Nemotron 3.5 Super VL (26.5, 120B), Beam 1.0 (30.3, 501B) and Muse Spark Open (38.9), the highest US open-weights score so far, and maps them against the Pareto frontier of size and intelligence.
Artificial Analysis finds Opus 4.7 effectively tied with Gemini 3.1 Pro and GPT-5.4 at the top of its index, leading on agentic knowledge work, with lower hallucination via more abstention and fewer output tokens than Opus 4.6.
Grok 4.6 (high) measured across the Intelligence Index: 61 points, sixth of 187 models in its class, with a 500k context window, $2/$6 per million tokens and a 75% cache discount — capable and mid-priced, but slower and wordier than most of its peers.
Artificial Analysis benchmarks Ling 3.1 Flash, a 560B/25B-active reasoning model with 1M context. It doubles its predecessor's index score and improves AA-Omniscience accuracy and hallucination rate. Open weights are expected soon.