Artificial Analysis
2 Tools · 276 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
DeepSeek V4.1 Flash 1017 scores 45 on the Artificial Analysis Intelligence Index, up 6 points. GDPval-AA Elo rises 146, non-hallucination rate goes from 4% to 33%, and the price stays at $0.30/$1.20 per 1M tokens. It is a 552B MoE with 16B active, MIT licensed, with a 1M context window.
Artificial Analysis introduces AA-PersonalityBench, scoring 38 models on 30 judged traits grouped into 8 indexes such as Warmth, Conviction and Slop, plus 21 programmatically measured communication characteristics like length and formatting.
Artificial Analysis reports HeyGen Voice at #1 on its Controlled Voice TTS Arena (Elo 1,201), with pricing, speed and pronunciation robustness compared against Qwen and ElevenLabs models.
Artificial Analysis publishes first AA-AgentPerf-Local results, replaying recorded agent trajectories on an NVIDIA RTX Spark laptop and comparing it to DGX Spark, Ryzen AI Halo, M5 Pro and an RTX 5090.
Artificial Analysis publishes its Terminal-Bench-Science 0.1 leaderboard, a 70-task agentic benchmark across five research domains. GPT-6 Astra (63%) and Claude Opus 5.5 (62%) lead, open models sit near 10%, and effort settings drive large score and cost differences.
Artificial Analysis benchmarks OpenAI's gpt-oss-120b (index 58) and gpt-oss-20b (48): MXFP4 weights of 61GB and 13GB, sparse MoE architecture, best intelligence runnable on one H100 or a consumer GPU, behind larger DeepSeek and Qwen models.
Upstage releases Solar Mini 4, a proprietary 35B/3B-active reasoning model scoring 24 on the Intelligence Index at $0.10/$0.40, strong on long context but weak at agentic coding and token-hungry.
Artificial Analysis launches AA-AnalystAgent: 80 private quantitative analysis tasks over real spreadsheets and documents, five runs each, scoring pass^5. Top models reach only about 50 to 54%.
Artificial Analysis benchmarked OpenAI's new GPT Image 2.5 Flare and Sunburst, finding they take the top two spots on both the Text-to-Image and Image-Editing leaderboards at GPT Image 2 pricing, with the biggest gains in composition/framing and text/symbol edits.
Artificial Analysis benchmarks MiniMax-M2.5, a 229B/10B MoE: Intelligence Index 42, stronger agentic performance, a hallucination regression, unchanged token usage, 200k context, modified MIT license and third-party API availability.
Artificial Analysis launches an Openness Index scoring models on availability (weights, licensing) and transparency (pre-training data, post-training methodology, code), arguing weights alone do not make a model open.
ITBench-AA tests models on 59 Kubernetes incident tasks, scoring root-cause precision at full recall. Opus 4.7 leads at 47%, GPT-5.5 follows at 46%, and no frontier model exceeds 50%.
Alibaba's Fun-Realtime-TTS takes first on the Artificial Analysis Speech Arena with Elo 1,219, priced at about $27.6 per million characters, with voice cloning and regional accent support.
Artificial Analysis adds DeepSeek V4.1 Flash to its agentic inference benchmark, reporting NVIDIA GB300 leading at 100 tokens/s per user, plus architecture notes on the model's Engram memory and sparse attention.
Artificial Analysis launches AA-PersonalityBench, scoring models on 30 personality traits rolled into eight indexes (including Warmth, Conviction and Slop) plus 21 measured style characteristics. It reports leaders per index and large cross-model differences in length, tables, em dashes and emoji.
Artificial Analysis reports that OpenAI's GPT-6 Sol and Luna roughly match their predecessors on its Intelligence Index while costing about half as much per task, mainly because token pricing fell around 50%. Chart details were not available.
Artificial Analysis finds Claude Opus 4.8 leads its index at 61.4, retakes GDPval-AA, improves Terminal-Bench Hard by 6.8 points, and keeps Opus 4.7 pricing and 1M context.
Artificial Analysis introduces the Decision Index v1.0, scoring decision models on knowledge, reasoning, When2Call tool-calling and WildGuard safety classification. It finds Jev leads decision models while small language models still win on accuracy at higher cost and latency.
Artificial Analysis benchmarks Kimi K3.1 Pro: 55 on its Intelligence Index, the top Terminal-Bench 4.0 score at 67%, 73% on AutomationBench-AA, a lower hallucination rate, $1.98 per task, and a narrower gap between open-weights and proprietary models.
Artificial Analysis scores Claude Opus 4.5 at 70 on its Intelligence Index, #2 behind Gemini 3 Pro, with the top Terminal-Bench Hard result, and shows the price cut lowered index cost from $3.1k to $1.5k despite 60% more tokens.
Artificial Analysis benchmarked Cognition's Devin Fusion, which pairs a frontier model with a cheaper SWE-2 sidekick model, finding the GPT-6 Astra pairing costs 43% less and runs 31% faster than the Claude Fable 5.1 pairing while scoring close behind it.
Artificial Analysis finds Kimi K2.5 the leading open-weights model: 1309 GDPval-AA Elo, first native image and video input for Moonshot, ~595GB INT4 weights, $371 cost to run the index, and a comparatively low hallucination rate.
Artificial Analysis measures time per task on AA-Briefcase: Opus 4.8 takes about 23 minutes while GPT-5.5 xhigh takes about 11. Tool execution accounts for roughly 12% of time, and GLM-5.2 leads open-weights models.
Artificial Analysis scores Kimi K3.1 Pro at 55 on its Intelligence Index, 4th overall, with large agentic gains: Terminal-Bench 4.0 rises from 13% to 67% and GDPval-AA by 312 Elo. Cost per task is about $1.98. Weights are not yet published.