Artificial Analysis
2 Tools · 299 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
Artificial Analysis benchmarks Cognition's Devin Fusion, which runs a frontier lead model with a cheaper sidekick in parallel. It scores 68.1 on the Coding Agent Index, ahead of Claude Code with Opus 5.5 at 66.0, at about 31% lower cost per task and fewer minutes. Results are mixed across the three
Artificial Analysis load-tests 8x MI300X, H100 and H200 systems on DeepSeek R1 and Llama 4 Maverick. It maps the latency versus throughput tradeoff across concurrency levels and discloses AMD sponsorship and methodology.
Artificial Analysis launches Code2Video-AA, its implementation of HeyGen's Code2Video Bench: 114 design briefs rendered through HyperFrames HTML and judged on five axes. It reports a model ranking, cost per task from $0.01 to $3.89, and a render success rate of at least 96%.
Artificial Analysis reports CyberGym-E2E-AA results on defensive memory-safety work, from discovery to patching. Some frontier models safety-block over 85% of tasks, while GPT-6 Luna and MiMo-V2.6-Pro run large-codebase bug hunts at up to 100x lower cost per task than Grok 4.7.
Artificial Analysis benchmarks DeepSeek's new Search API as an agent web_search tool: index score 69 (17th of 27), cheapest per search, but weak on multi-hop BrowseComp, heavy 7k-token results, a 128-character query cap, and 5 searches per second.
Artificial Analysis benchmarks Xiaomi's MiMo-V2-Pro: 49 on its index, leading open-weights peers on GDPval-AA, low hallucination, 1M context, cheap pricing, but no released weights.
Artificial Analysis benchmarks Meta's Muse Spark 1.3: 61 (xhigh) and 62 (max, partner preview) on its Intelligence Index, gains concentrated in agentic evals, small regressions in long-context and accuracy, and the lowest cost per task among models scoring 59 or higher.
Artificial Analysis introduces AA-LCR, a benchmark of 100 questions over roughly 100k-token document sets. Top models score about 69%, and the results differ from needle-in-haystack tests and general reasoning rankings.
AA-AgentPerf replays real coding-agent trajectories to find how many concurrent agents a system sustains within speed and latency targets per megawatt. Initial NVIDIA and AMD results show Blackwell leading Hopper.
Artificial Analysis benchmarks Grok 4.7: Intelligence Index 46, Coding Agent Index 56 (4th with Grok Build), frontier-level agentic knowledge work Elo, and much higher token use per task. Pricing stays at $2/$6 per 1M tokens with a 500k context window.
Artificial Analysis benchmarks Meta's Muse Wonder (52) and Muse Wonder Full (56) on Intelligence Index v4.3, up from Muse Spark 1.3 at 48. Agentic evals improve most. Covers token use, $4/$20 per million token pricing, 1M context, and limited availability of the Full model.
Artificial Analysis covers DeepSeek V4 Pro (1.6T, 49B active) and V4 Flash, scoring 52 and 47, leading open weights on agentic work but with hallucination rates above 90%.
Artificial Analysis benchmarks Ant Group's Ling-3.0-flash-Fin, an MIT-licensed 124B MoE (5.1B active) finance model. It ties MiniMax-M2.7 at 23 on the Intelligence Index but burns more output tokens and struggles on hard agentic tasks.
Artificial Analysis benchmarks OpenBMB's MiniCPM5-2B, a 2.6B dense Apache 2.0 reasoning model scoring 15 on the Intelligence Index, the best under 4B parameters, using 19k tokens per task.
Artificial Analysis reports Liquid AI's d1-ultra leading its Decision Index at 65. It abstains far more often (85% non-hallucination rate), trading accuracy (32%) for calibration, at 92 ms per task and $32 per 1M tasks.
Artificial Analysis benchmarks Krea 3 in Fast and Ultra modes on its image editing arena: Ultra ranks #5 overall and #2 in Animation & Gaming and Frontier, Fast ranks #11. Includes API pricing and per-action ranking differences between the modes.
Artificial Analysis scores GLM-5.3 at 60 on Index v4.1, tied with Kimi K3 among open-weights models. Agentic Elo jumps 246 points, but token use rises about 20 percent, cost per task is $0.68, and hallucination rate ticks up to 30 percent. Later v5 rescoring is lower.
Google DeepMind releases Gemini 3.8 Flash TTS and Flash-Lite TTS, debuting at #2 and #6 on Artificial Analysis's Provider Voice Arena and #1 on pronunciation robustness, processing 44.1 and 40.2 characters per second respectively.
Artificial Analysis benchmarks DeepSeek V4 Pro 0813: 53 on its Intelligence Index, 1.6T parameters with 49B active, MIT weights, and a 1M context. New pricing of $1.32 in and $3.96 out lifts cost per task fivefold, leaving it only a point ahead of V4 Flash, with gains in agentic work and token effic
Artificial Analysis charts GPT-5.6 Sol, Terra and Luna on intelligence versus cost per task and finds Terra is behind Luna or Sol at every point, with Luna especially cost efficient.
Artificial Analysis finds GPT-6.1 Sol extends the cost-efficiency frontier, at $0.72 per Intelligence Index task at max effort, 31% below GPT-6 Sol. A fixed image-encoding bug also raised GPT-6 Luna by a point on vision-heavy evals.
Artificial Analysis benchmarks Grok 4.6: index 61, GDPval-AA 1753 Elo, 88.4% Terminal-Bench, unchanged $2/$6 pricing and $0.84 cost per task on the Pareto frontier.
Microsoft's MAI-Transcribe-1.5 ranks third on word error rate at 2.4%, runs at about 276x real time, adds keyword biasing, and supports 43 languages at $6 per 1,000 minutes.
Artificial Analysis benchmarks Google's Nano Banana 2.1, built on Gemini 3.6 Flash. It ranks #4 on text-to-image and editing, improves on its predecessor across all 16 measured capabilities, and costs $0.0336 per 1K image, the lowest among the top seven models.