Artificial Analysis
2 Tools · 276 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
Artificial Analysis benchmarks Google's Gemini 3.6 Flash and 3.5 Flash-Lite: both cut time per task by about half, Flash holds intelligence at 50 with cheaper output pricing, and Flash-Lite jumps 11 points while its per-task cost doubles.
Mistral releases Voxtral 3, with a TTS model and three speech-to-text models. Artificial Analysis ranks Small Transcribe 3 third at 2.1% WER, reports realtime latency and WER at two settings, and places the TTS model twelfth in its arenas at $13 per 1M characters.
MiniMax-M3 scores 55 on the Artificial Analysis index with image and video input and a 1M context. Weights are promised within about ten days, and heavy abstention drives its low hallucination rate.
Artificial Analysis benchmarks OpenAI's integrated web search tool against 25 other search setups using one fixed model. It ranks 7th of 26, is strongest on factual accuracy and weaker on multi-hop BrowseComp, with cost per task compared.
Artificial Analysis compares 14 models on MLCR-AA and HealthBench Professional. Anthropic models lead on completeness and raw score, OpenAI models on accuracy and concision, and length-adjusted scoring closes the gap, with GPT-6 Astra tying Sonnet 5.5 for first.
AutomationBench-AA scores agents on 657 SaaS workflow tasks, counting objectives met without guardrail violations. Top scores are about 49%, every model breaks rules, and Gemini 3.5 Flash matches GPT-5.5 at roughly 37% of the cost.
Artificial Analysis places Ideogram 4.5 at #23 in image editing and #34 in text-to-image. High quality costs $0.22 per edit and $0.10 per generation, lower tiers start at $0.008, and it is on the Ideogram API with open weights promised.
Artificial Analysis finds Gemini 3.8 Flash TTS leads its Pronunciation Robustness benchmark at 89.5%, ahead of prior Gemini and SpaceXAI models, with particular strength on contextual disambiguation and shorthand expansion.
Artificial Analysis finds Gemini 3 Pro Preview first on its Intelligence Index, 3 points above GPT-5.1, leading five of ten evals including HLE and AA-Omniscience, with better token efficiency than rivals but high per-token cost.
Artificial Analysis publishes a leaderboard for Cognition's FrontierCode 1.1, a private benchmark of maintainer-grade open source changes. Scoring combines unit tests with merge-readiness criteria like diff size and scope, run through a shared harness.
Artificial Analysis runs Harvey's legal agent benchmark on 120 private tasks. Claude Fable 5 leads with a 14.2% all-pass rate, and many models fully pass none despite high per-criterion scores.
Z AI's GLM-5.3 scores 60 on the Artificial Analysis Intelligence Index, ninth of 187 and seven points above GLM-5.2, with a 1M context at $1.40/$4.40 per million tokens. It is faster than the median at 77 tok/s but unusually verbose, spending 170M output tokens on the index.