Artificial Analysis
2 Tools · 276 Intel
Artificial Analysis is an independent AI benchmarking organization, founded in 2023 by Micah Hill-Smith and George Cameron, that runs its own evaluations of AI models and inference providers rather than relying on self-reported figures from AI labs. Its free public leaderboards and Intelligence Index have become widely referenced industry benchmarks for comparing model quality, speed, and cost. The company is backed by investors including Nat Friedman, Daniel Gross, and Andrew Ng.
Is this you? Sign in with X to claim this profile.
Tools
Independent benchmarks comparing AI models and providers on intelligence, coding, speed, latency, and price.
Artificial Analysis's platform for creating and running custom benchmarks against AI models on your own tasks.
Intel
Grok 4.5 scores 54 on the Artificial Analysis index, tops tau3-Banking and, in Grok Build, rivals GPT-5.5 on coding-agent tests at a fraction of the cost and token use.
Artificial Analysis releases Intelligence Index v4.3, swapping in Terminal-Bench 4.0 and AutomationBench-AA. Claude Fable 5.1 and GPT-6 Astra tie at 53, and GLM-5.3 and Kimi K3 lead open weights.
Artificial Analysis benchmarks 15 search APIs from 7 providers inside a fixed agent harness — same model, same tasks, only the search tool varies — across DeepSearchQA, BrowseComp and AA-Omniscience, with cost and time per task. Parallel and Firecrawl hold the cost-quality frontier.
OpenBMB's MiniCPM-V 4.6 1.3B scores 13 on the Artificial Analysis index, best under 2B parameters, supports text, image and video input, and used only 5.4M output tokens to run the benchmark.
Artificial Analysis benchmarks Grok 4.7: 46 on the Intelligence Index, a 56 Coding Agent Index with Grok Build, large AA-Briefcase gains, and about 81k output tokens per task at $2/$6 pricing.
Artificial Analysis reports HiDream-O1-Video-1.0 debuting sixth in its image-to-video-with-audio arena, with 1080p output of 5 to 20 seconds, native audio, and API pricing of $5.80 per minute.
Artificial Analysis benchmarks Upstage's Solar Mini 4, a 35B/3B-active proprietary reasoning model. It scores 24 on the Intelligence Index with strong long-context results, but heavy token use inflates per-task cost and agentic coding is weak.
Artificial Analysis ranks Meta Superintelligence Labs' first image model #4 in image editing and #5 in text to image, on the price/quality Pareto frontier. Muse Image invokes search and coding tools and self-refines, and reaches developers via the Meta Model API, fal, Runway, and OpenRouter.
OpenBMB's MiniCPM5-1B scores 17.9 on the Artificial Analysis index, the best under 2B parameters, using far fewer tokens than reasoning peers and abstaining rather than hallucinating.
Artificial Analysis launches AA-Briefcase, a private benchmark of multi-week knowledge-work projects graded on rubric pass rate, analytical quality and presentation. Claude Fable 5 leads, with GLM-5.2 the top open-weights model.
Artificial Analysis publishes Cognition's FrontierCode 1.1 results: private tasks from 36 open source repos graded on maintainer-merge criteria. Claude Opus 5.5 leads with a 58% pass rate.
Artificial Analysis opened Optima: hand it your tasks and example artifacts, and a build agent drafts tasks plus grading rubrics, then runs frontier models — or your own agent over HTTP — on the identical set, reporting score against cost and time per task.
Artificial Analysis ranks Inworld's Realtime TTS-2 first on its Controlled Voice Arena (1,123 Elo) and second on the Provider Voice Arena, with 106 characters per second throughput and $20.83 per 1M characters pricing, plus 100+ language support and inline delivery instructions.
Artificial Analysis finds GLM-5 (744B/40B MoE) the first open-weights model to reach 50 on its Intelligence Index, with a GDPval-AA Elo of 1412, sharply reduced hallucination and lower token use than GLM-4.7.
Artificial Analysis benchmarks the generally available Gemini 4 Argon. It ties GPT-6 Astra on the Intelligence Index at lower cost per task, improves on agentic evals, posts a 15% hallucination rate, and is priced at half rate until Nov 30 before doubling.
Artificial Analysis's new Pronunciation Robustness benchmark tests whether TTS models correctly voice context-dependent words, shorthand, and exact sequences like emails or codes; Gemini 3.1 Flash TTS leads at 88.1%, ahead of SpaceXAI TTS (87.6%) and ElevenLabs Eleven v3 (85.6%).
Artificial Analysis benchmarks Meta's Muse Glimmer, a 30B Apache 2.0 open-weights model scoring 35 on the index, with memory footprints for self-hosting and weak GDPval-AA agentic results.
New dashboard pages let you compare intelligence, cost, and latency across every reasoning/effort configuration of a model, since frontier releases now ship with up to six effort tiers with very different cost-performance profiles.
Artificial Analysis benchmarks Claude Haiku 5.5 against small models, noting tiered pricing and heavy token use versus GPT-6 Luna. It flags that cost-per-task figures do not yet include the long-prompt price step.
Artificial Analysis benchmarks five frontier models on eight tabletop robot-arm tasks, with each model writing code against an HTTP API and four cameras. Top score is 69, with ~100x human time and ~10x cost, and common failures are missed grasps and misjudged completion.
Artificial Analysis benchmarks MiniMax-M3.1-Flash: Intelligence Index 42 (up 13 from M3) at about $0.22 per task, with strong HLE and AutomationBench results, weaker Terminal-Bench, and a 93% hallucination rate on unknown questions. Open weights are announced for October 23.
Artificial Analysis adds AMD's MI355X to its AA-AgentPerf benchmark, which replays real coding-agent sessions with caching and tool delays. Results compare it with B200 and B300 across Kimi K3, GLM and DeepSeek models.
Artificial Analysis introduces Stirrup, an open-source Python agent framework that lets models choose their approach. It bundles web tools, code execution, MCP client, automatic context summarization and multimodal handling, and powers their GDPval-AA benchmark.
Artificial Analysis chained 30 edits through four image editing models. Ideogram 4.5 and FLUX 3 edit locally and preserve the scene, Nano Banana 2.1 darkens gradually, and GPT Image 2.5 re-renders most of the frame each turn, so drift compounds.