Artificial Analysis
2 Tools · 299 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
Artificial Analysis benchmarks NVIDIA's Nemotron 3.5 Transcribe, a 30B total, 4B active open-weight speech-to-text model. It ties for first among open models at 2.8% WER, leads on voice agent conversations, and runs at 75x real time on DeepInfra for $2 per 1,000 minutes. Only the opening of the thre
Artificial Analysis added a Throughput per $/hr metric to AA-AgentPerf, using Silicon Data's daily rental indices for H100, H200, B200 and B300. B300 generally leads on value, and the MI355X price is an assumed figure.
Artificial Analysis publishes Cognition's FrontierCode 1.1 results: private tasks from 36 open source repos graded on maintainer-merge criteria. Claude Opus 5.5 leads with a 58% pass rate.
Artificial Analysis launched Optima, a GA platform for building custom benchmarks from your own files, traces or coding-agent context. It runs leading models, grades with rubrics or pairwise judges, and reports score alongside cost and time per task, with per-unit grading prices.
Artificial Analysis ranks Inworld's Realtime TTS-2 first on its Controlled Voice Arena (1,123 Elo) and second on the Provider Voice Arena, with 106 characters per second throughput and $20.83 per 1M characters pricing, plus 100+ language support and inline delivery instructions.
Artificial Analysis releases Intelligence Index v5 with FrontierCode 2.0 and Terminal-Bench-Science, upgraded HLE, Omniscience and CritPt, and 60% private evaluation. Claude Opus 5.5 leads at 58, the open-weights gap grows to 12 points, and cost per task varies over 13x.
Artificial Analysis finds GLM-5 (744B/40B MoE) the first open-weights model to reach 50 on its Intelligence Index, with a GDPval-AA Elo of 1412, sharply reduced hallucination and lower token use than GLM-4.7.
Artificial Analysis benchmarks the generally available Gemini 4 Argon. It ties GPT-6 Astra on the Intelligence Index at lower cost per task, improves on agentic evals, posts a 15% hallucination rate, and is priced at half rate until Nov 30 before doubling.
Artificial Analysis introduces a human-judged TTS benchmark covering context-dependent words, shorthand, exact sequences, and standalone terms. Gemini 3.1 Flash TTS leads at 88.1%, and listener preference does not predict pronunciation accuracy.
Artificial Analysis benchmarks Meta's Muse Glimmer, a 30B Apache 2.0 open-weights model scoring 35 on the index, with memory footprints for self-hosting and weak GDPval-AA agentic results.
Artificial Analysis added Model Release pages that compare intelligence, cost per task, output speed and latency across every effort level of a model, with capability index scores by domain and side-by-side comparison against similar models.
Artificial Analysis benchmarks Claude Haiku 5.5: Intelligence Index 43, ahead of other small models, but with ~162k output tokens per task at max effort. Covers tiered pricing, Terminal-Bench 4.0 gains, hallucination rate and a likely understated AutomationBench result.
Artificial Analysis benchmarks five frontier models on eight tabletop robot-arm tasks, with each model writing code against an HTTP API and four cameras. Top score is 69, with ~100x human time and ~10x cost, and common failures are missed grasps and misjudged completion.
Artificial Analysis benchmarks MiniMax-M3.1-Flash: Intelligence Index 42 (up 13 from M3) at about $0.22 per task, with strong HLE and AutomationBench results, weaker Terminal-Bench, and a 93% hallucination rate on unknown questions. Open weights are announced for October 23.
Artificial Analysis adds AMD's MI355X to its AA-AgentPerf benchmark, which replays real coding-agent sessions with caching and tool delays. Results compare it with B200 and B300 across Kimi K3, GLM and DeepSeek models.
Artificial Analysis introduces Stirrup, an open-source Python agent framework that lets models choose their approach. It bundles web tools, code execution, MCP client, automatic context summarization and multimodal handling, and powers their GDPval-AA benchmark.
Artificial Analysis chained 30 edits through four image editing models. Ideogram 4.5 and FLUX 3 edit locally and preserve the scene, Nano Banana 2.1 darkens gradually, and GPT Image 2.5 re-renders most of the frame each turn, so drift compounds.
DeepSeek V4.1 Flash 1017 scores 45 on the Artificial Analysis Intelligence Index, up 6 points. GDPval-AA Elo rises 146, non-hallucination rate goes from 4% to 33%, and the price stays at $0.30/$1.20 per 1M tokens. It is a 552B MoE with 16B active, MIT licensed, with a 1M context window.
Artificial Analysis introduces AA-PersonalityBench, scoring 38 models on 30 judged traits grouped into 8 indexes such as Warmth, Conviction and Slop, plus 21 programmatically measured communication characteristics like length and formatting.
Artificial Analysis reports AA-PersonalityBench results for Grok 4.5, 4.6 and 4.7. Slop dropped from the 68th to the 6th percentile, sycophancy from 18 to 1, and sincerity rose from 26 to 90. Methodology was linked but not reviewed.
Artificial Analysis places HeyGen Voice first in its Controlled Voice TTS Arena (Elo 1,201), ahead of Qwen and ElevenLabs models. It also lists pricing ($30 per 1M characters), generation speed and pronunciation robustness, where it ranks tenth of 29.
Artificial Analysis publishes first AA-AgentPerf-Local results, replaying recorded agent trajectories on an NVIDIA RTX Spark laptop and comparing it to DGX Spark, Ryzen AI Halo, M5 Pro and an RTX 5090.
Artificial Analysis publishes its Terminal-Bench-Science 0.1 leaderboard, a 70-task agentic benchmark across five research domains. GPT-6 Astra (63%) and Claude Opus 5.5 (62%) lead, open models sit near 10%, and effort settings drive large score and cost differences.
Artificial Analysis benchmarks OpenAI's gpt-oss-120b (index 58) and gpt-oss-20b (48): MXFP4 weights of 61GB and 13GB, sparse MoE architecture, best intelligence runnable on one H100 or a consumer GPU, behind larger DeepSeek and Qwen models.