Artificial Analysis
2 Tools · 276 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
Artificial Analysis benchmarks Ant Group's Ling-3.0-flash-Fin, an MIT-licensed 124B MoE (5.1B active) finance model. It ties MiniMax-M2.7 at 23 on the Intelligence Index but burns more output tokens and struggles on hard agentic tasks.
Artificial Analysis benchmarks OpenBMB's MiniCPM5-2B, a 2.6B dense Apache 2.0 reasoning model scoring 15 on the Intelligence Index, the best under 4B parameters, using 19k tokens per task.
Artificial Analysis reports Liquid AI's d1-ultra leading its Decision Index at 65. It abstains far more often (85% non-hallucination rate), trading accuracy (32%) for calibration, at 92 ms per task and $32 per 1M tasks.
Artificial Analysis benchmarks Krea 3 in Fast and Ultra modes on its image editing arena: Ultra ranks #5 overall and #2 in Animation & Gaming and Frontier, Fast ranks #11. Includes API pricing and per-action ranking differences between the modes.
Artificial Analysis scores GLM-5.3 at 60 on its intelligence index, level with Kimi K3, with a 246 point jump to 1770 Elo on its agentic evaluation. It burns about 20 percent more output tokens than GLM-5.2 and costs 68 cents per index task against 44 cents.
Google DeepMind releases Gemini 3.8 Flash TTS and Flash-Lite TTS, debuting at #2 and #6 on Artificial Analysis's Provider Voice Arena and #1 on pronunciation robustness, processing 44.1 and 40.2 characters per second respectively.
Artificial Analysis's evaluation of DeepSeek V4 Pro 0813: 53 on the Intelligence Index, fifth of 110 models, with 1.6T total and 49B active parameters, MIT-licensed weights and a 1M context. Pricing of $1.32 in and $3.96 out lifts cost per task fivefold, narrowing its edge over V4 Flash to a point.
Artificial Analysis charts GPT-5.6 Sol, Terra and Luna on intelligence versus cost per task and finds Terra is behind Luna or Sol at every point, with Luna especially cost efficient.
Artificial Analysis reports GPT-6.1 Sol extends the cost-efficiency Pareto frontier, at 31% below GPT-6 Sol per task. An image-encoding fix also lifted GPT-6 Luna by one Intelligence Index point, mostly on vision-heavy evals.
Artificial Analysis benchmarks Grok 4.6: index 61, GDPval-AA 1753 Elo, 88.4% Terminal-Bench, unchanged $2/$6 pricing and $0.84 cost per task on the Pareto frontier.
Microsoft's MAI-Transcribe-1.5 ranks third on word error rate at 2.4%, runs at about 276x real time, adds keyword biasing, and supports 43 languages at $6 per 1,000 minutes.
Artificial Analysis evaluates Google's Nano Banana 2.1 (released Oct 6, built on Gemini 3.6 Flash). It ranks #4 in text-to-image and editing, beats its predecessor in all 16 capabilities, and costs $0.0336 per 1K image, the lowest among top-7 models.
Artificial Analysis benchmarks Gemini 3.7 Flash: index 56, Pareto frontier on time per task, discounted pricing, and top pass^5 on AA-AnalystAgent.
Artificial Analysis measures Claude Opus 5.5 at 66 on its Coding Agent Index in Claude Code at max effort, up from 60 for Opus 5. Cost per task rises to $13.04 as token use climbs to 15.6M per task, outweighing lower per-token prices.
Artificial Analysis rates Ant Group's open-weights Ming-3.0-tts as #1 open model on Controlled Voice and Pronunciation Robustness (93.2%). Weights for the 0.5B dense and 7.9B MoE variants are on Hugging Face under MIT, with voice cloning included.
Artificial Analysis finds DeepSeek V4.1 Flash 1017 the top publicly available model on its Cyber Index at 57%, up 16 points on its predecessor, driven by CyberGym-E2E-AA jumping from 23% to 76%. Cost is $0.42 per task and it triggers no safety refusals.
Artificial Analysis evaluates Gemini 3.8 Flash: 59 on the Intelligence Index, $0.58 per task at unchanged $0.75/$3.75 pricing, and agentic gains driven by 30% more output tokens per task.
Artificial Analysis places Ideogram 4.5 at #23 on its image editing leaderboard and #34 on text to image, with pricing from $0.008 to $0.22 per image by quality tier. It is live in the API, with open weights promised.
Artificial Analysis finds Gemini 4 Argon matching GPT-6 Astra at 53 on the Intelligence Index, with 57% on Terminal-Bench 4 and a 15% hallucination rate. It costs $1.99 per task under a discount that doubles later.
Artificial Analysis reports Claude Sonnet 4.6 as the new GDPval-AA leader at 1633 Elo, statistically tied with Opus 4.6. It reached that with over 4x the tokens of Sonnet 4.5, which pushed total benchmark cost above Opus 4.6.
Cohere's North Mini Code is a 30B (3B active) Apache-2.0 coding model scoring 27.6 on the Artificial Analysis index and 33.4 on coding, at about 199 tokens per second, but only 14% on GDPval-AA.
Grok 4.5, GPT-5.6, Muse Spark 1.1 and Kimi K3 launched within eight days. Artificial Analysis shows six labs now above 50 on its index and near-frontier cost per task falling two to three times.
StepFun's 600B-parameter Step 5 Preview scores 44 on Artificial Analysis's Intelligence Index, tying Kimi K3 (max) at about 2.8x lower cost per task, with large reasoning gains on HLE and CritPt but weaker results on agentic evaluations than similarly scored peers.
StepFun's StepAudio 3 ASR tops Artificial Analysis's non-streaming speech-to-text leaderboard at 1.7% word error rate, though it transcribes slower and costs more per minute than several rivals.