Announcing Artificial Analysis Capability Indices v1.1, updated with stronger domain tuning, combining slices of core Intelligence Index v4.3 evaluations alongside specialized evaluations
The Capability Indices map tasks from O*NET occupations to benchmarks that represent them, weighting each benchmark by how often its capability appears across the work. We cover six indices: Finance & Accounting, Strategy & Ops, Legal, Healthcare & Medical, Engineering, and Economics
Key changes:
➤ Finance & Accounting: Agentic Tool Use added as a new capability sourced from AutomationBench-AA (Finance), Agentic Customer Interaction removed from capabilities, GDP.pdf added to Long-Context Reasoning, and AA-Briefcase added to Agentic Knowledge Work
➤ Strategy & Ops: Agentic Tool Use added as a new capability sourced from AutomationBench-AA (Operations), Agentic Customer Interaction removed from capabilities, GDP.pdf added to Long-Context Reasoning, and AA-Briefcase added to Agentic Knowledge Work
➤ Legal: Agentic Tool Use added as a new capability sourced from AutomationBench-AA (Operations and Support), Agentic Customer Interaction removed from capabilities, GDP.pdf added to Long-Context…
tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
If you use Artificial Analysis indices to pick models for finance, legal, healthcare or ops work, the v1.1 reweighting (new agentic tool-use and long-context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → benchmarks) can shift which model looks best for your domain.
Read the full post on X ↗
postAA benchmarks Devin Fusion, first multi-model coding agent on its index
postLing-3.0-flash-VL leads on efficiency, lags on agentic benchmarks, AA finds
postOcten Search Ranks Third on Artificial Analysis's Search Agent Index
