Artificial Analysis
2 Tools · 299 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
AssemblyAI launched its first text-to-speech model. Artificial Analysis ranks it #12 in its Provider Voice Arena and #10 in Controlled Voice, scores 81.7% on pronunciation robustness, and measures 162 characters per second at $15 per 1M characters.
Upstage releases Solar Mini 4, a proprietary 35B/3B-active reasoning model scoring 24 on the Intelligence Index at $0.10/$0.40, strong on long context but weak at agentic coding and token-hungry.
Artificial Analysis launches AA-AnalystAgent: 80 private quantitative analysis tasks over real spreadsheets and documents, five runs each, scoring pass^5. Top models reach only about 50 to 54%.
Artificial Analysis released Intelligence Index v5, adding FrontierCode 2.0 and Terminal-Bench-Science, upgrading three evaluations, and moving 60% of weighting to private test sets. It also launched a Progress Index for cross-time comparison. Claude Opus 5.5 leads at launch.
Artificial Analysis benchmarks Thinking Machines' Inkling 2 Small (594B total, 17B active): 31 on the Intelligence Index, large agentic gains, lower hallucination rate, roughly 4x the output tokens, and $0.30/$1.00 pricing. Available on Tinker; open weights promised later.
Artificial Analysis benchmarks OpenAI's GPT Image 2.5 Flare and Sunburst, which take the top two spots on both its text-to-image and image-editing arenas at GPT Image 2 pricing, with largest gains in composition and framing and text or symbol edits.
Artificial Analysis benchmarks MiniMax-M2.5, a 229B/10B MoE: Intelligence Index 42, stronger agentic performance, a hallucination regression, unchanged token usage, 200k context, modified MIT license and third-party API availability.
Artificial Analysis launches an Openness Index scoring models on availability (weights, licensing) and transparency (pre-training data, post-training methodology, code), arguing weights alone do not make a model open.
ITBench-AA tests models on 59 Kubernetes incident tasks, scoring root-cause precision at full recall. Opus 4.7 leads at 47%, GPT-5.5 follows at 46%, and no frontier model exceeds 50%.
Alibaba's Fun-Realtime-TTS takes first on the Artificial Analysis Speech Arena with Elo 1,219, priced at about $27.6 per million characters, with voice cloning and regional accent support.
Artificial Analysis adds DeepSeek V4.1 Flash to its agentic inference benchmark. GB300 leads at 100 tok/s per user, ahead of B300, B200, GB200 and MI355X. Engram memory and compressed sparse attention shrink the KV cache, so more concurrent agents fit per GPU.
Artificial Analysis launches AA-PersonalityBench, scoring models on 30 personality traits rolled into eight indexes (including Warmth, Conviction and Slop) plus 21 measured style characteristics. It reports leaders per index and large cross-model differences in length, tables, em dashes and emoji.
Artificial Analysis reports that OpenAI's GPT-6 Sol and Luna roughly match their predecessors on its Intelligence Index while costing about half as much per task, mainly because token pricing fell around 50%. Chart details were not available.
Artificial Analysis finds Claude Opus 4.8 leads its index at 61.4, retakes GDPval-AA, improves Terminal-Bench Hard by 6.8 points, and keeps Opus 4.7 pricing and 1M context.
Artificial Analysis introduces the Decision Index v1.0, scoring decision models on knowledge, reasoning, tool-calling and safety classification. Jev 1.13 leads at 58, while Claude Haiku 5.5 scores 61 at 11x the latency and 4.4x the cost. Calibration is weak on knowledge and reasoning.
Artificial Analysis benchmarks Kimi K3.1 Pro: 55 on its Intelligence Index, the top Terminal-Bench 4.0 score at 67%, 73% on AutomationBench-AA, a lower hallucination rate, $1.98 per task, and a narrower gap between open-weights and proprietary models.
Artificial Analysis scores Claude Opus 4.5 at 70 on its Intelligence Index, #2 behind Gemini 3 Pro, with the top Terminal-Bench Hard result, and shows the price cut lowered index cost from $3.1k to $1.5k despite 60% more tokens.
Artificial Analysis added Cognition's Devin Fusion to its Coding Agent Index, the first multi-model agent listed. Pairing Claude Fable 5.1 with the SWE-2 sidekick scores 62. Pairing GPT-6 Astra scores 59 at 43% lower cost and 31% faster completion.
Artificial Analysis finds Kimi K2.5 the leading open-weights model: 1309 GDPval-AA Elo, first native image and video input for Moonshot, ~595GB INT4 weights, $371 cost to run the index, and a comparatively low hallucination rate.
Artificial Analysis measures time per task on AA-Briefcase: Opus 4.8 takes about 23 minutes while GPT-5.5 xhigh takes about 11. Tool execution accounts for roughly 12% of time, and GLM-5.2 leads open-weights models.
Artificial Analysis scores Kimi K3.1 Pro at 55 on its Intelligence Index, 4th overall, with large agentic gains: Terminal-Bench 4.0 rises from 13% to 67% and GDPval-AA by 312 Elo. Cost per task is about $1.98. Weights are not yet published.
Artificial Analysis benchmarks Google's Gemini 3.6 Flash and 3.5 Flash-Lite: both cut time per task by about half, Flash holds intelligence at 50 with cheaper output pricing, and Flash-Lite jumps 11 points while its per-task cost doubles.
Mistral releases Voxtral 3, with a TTS model and three speech-to-text models. Artificial Analysis ranks Small Transcribe 3 third at 2.1% WER, reports realtime latency and WER at two settings, and places the TTS model twelfth in its arenas at $13 per 1M characters.
Artificial Analysis benchmarks GitHub Copilot Fusion, which orchestrates Anthropic and OpenAI models. It scores near Claude Code with Opus 5.5 on the Coding Agent Index at about half the cost per task and roughly half the tokens.