Artificial Analysis
2 Tools · 299 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
Artificial Analysis benchmarks GPT-6.1 Terra: half the per-token price of GPT-5.6 Terra, 6 more Intelligence Index points at under a third of the per-task cost, a 50-point drop in hallucination rate, and a roughly 20-point jump on Terminal-Bench 4.0.
Artificial Analysis benchmarks Gemini 3.7 Flash: index 56, Pareto frontier on time per task, discounted pricing, and top pass^5 on AA-AnalystAgent.
Artificial Analysis measures Claude Opus 5.5 at 66 on its Coding Agent Index in Claude Code at max effort, up from 60 for Opus 5. Cost per task rises to $13.04 as token use climbs to 15.6M per task, outweighing lower per-token prices.
Artificial Analysis rates Ant Group's open-weights Ming-3.0-tts as #1 open model on Controlled Voice and Pronunciation Robustness (93.2%). Weights for the 0.5B dense and 7.9B MoE variants are on Hugging Face under MIT, with voice cloning included.
Artificial Analysis finds DeepSeek V4.1 Flash 1017 the top publicly available model on its Cyber Index at 57%, up 16 points on its predecessor, driven by CyberGym-E2E-AA jumping from 23% to 76%. Cost is $0.42 per task and it triggers no safety refusals.
Artificial Analysis evaluates Gemini 3.8 Flash: 59 on the Intelligence Index, $0.58 per task at unchanged $0.75/$3.75 pricing, and agentic gains driven by 30% more output tokens per task.
Artificial Analysis places Ideogram 4.5 at #23 on its image editing leaderboard and #34 on text to image, with pricing from $0.008 to $0.22 per image by quality tier. It is live in the API, with open weights promised.
Artificial Analysis finds Gemini 4 Argon matching GPT-6 Astra at 53 on the Intelligence Index, with 57% on Terminal-Bench 4 and a 15% hallucination rate. It costs $1.99 per task under a discount that doubles later.
Artificial Analysis reports Claude Sonnet 4.6 as the new GDPval-AA leader at 1633 Elo, statistically tied with Opus 4.6. It reached that with over 4x the tokens of Sonnet 4.5, which pushed total benchmark cost above Opus 4.6.
Cohere's North Mini Code is a 30B (3B active) Apache-2.0 coding model scoring 27.6 on the Artificial Analysis index and 33.4 on coding, at about 199 tokens per second, but only 14% on GDPval-AA.
Grok 4.5, GPT-5.6, Muse Spark 1.1 and Kimi K3 launched within eight days. Artificial Analysis shows six labs now above 50 on its index and near-frontier cost per task falling two to three times.
Artificial Analysis benchmarks StepFun's 600B-parameter Step 5 Preview: it ties Kimi K3 (max) at 44 on the Intelligence Index for about 2.8x less per task, with strong HLE and CritPt gains but lagging agentic scores.
StepFun's StepAudio 3 ASR tops Artificial Analysis's non-streaming speech-to-text index at 1.7% WER, tied with Alibaba's preview model. It is slower than several rivals and the priciest of the top five, but leads on voice-agent speech.
Grok 4.5 scores 54 on the Artificial Analysis index, tops tau3-Banking and, in Grok Build, rivals GPT-5.5 on coding-agent tests at a fraction of the cost and token use.
Artificial Analysis releases Intelligence Index v4.3, swapping in Terminal-Bench 4.0 and AutomationBench-AA. Claude Fable 5.1 and GPT-6 Astra tie at 53, and GLM-5.3 and Kimi K3 lead open weights.
Artificial Analysis launches a Search Index that swaps search APIs inside a fixed agent harness, scoring quality, cost and time on DeepSearchQA, BrowseComp and AA-Omniscience. Search lifts the model baseline of 33 to 65-75, and focused results can lower total cost per task.
OpenBMB's MiniCPM-V 4.6 1.3B scores 13 on the Artificial Analysis index, best under 2B parameters, supports text, image and video input, and used only 5.4M output tokens to run the benchmark.
Artificial Analysis benchmarks Grok 4.7: 46 on the Intelligence Index, a 56 Coding Agent Index with Grok Build, large AA-Briefcase gains, and about 81k output tokens per task at $2/$6 pricing.
HiDream-O1-Video-1.0 enters the Artificial Analysis image-to-video-with-audio arena at sixth place. It outputs 1080p clips of 5 to 20 seconds with synchronized audio and is sold through the HiHarness API at about $0.10 per second.
Artificial Analysis benchmarks Upstage's proprietary Solar Mini 4 (35B total, 3B active). It leads on long-context reasoning and has a low hallucination rate, but heavy token use inflates per-task cost and agentic coding is very weak.
Artificial Analysis ranks Meta's Muse Image #4 for image editing and #5 for text to image, on the quality/price Pareto frontier. The model uses search and coding tools and self-refines, and is available via the Meta Model API, fal, Runway and OpenRouter.
OpenBMB's MiniCPM5-1B scores 17.9 on the Artificial Analysis index, the best under 2B parameters, using far fewer tokens than reasoning peers and abstaining rather than hallucinating.
Artificial Analysis adds HLE-Diamond, the cleaned 1,000-question Humanity's Last Exam, to its index. Gemini 4 Carbon leads at 69%, models do better on reasoning than knowledge, open-weight models trail by over 30 points, and top effort settings multiply cost per task.
Artificial Analysis launches AA-Briefcase, a private benchmark of multi-week knowledge-work projects graded on rubric pass rate, analytical quality and presentation. Claude Fable 5 leads, with GLM-5.2 the top open-weights model.