We have updated the Artificial Analysis Intelligence Index to v4.1.1 - this patc

Many scores on the Artificial Analysis Intelligence Index rose in early August without any model changing, because the benchmark changed how it marks answers. Artificial Analysis, an independent benchmarking firm, combines a set of evaluations into the Index. Several of them accept free-form answers, so a second model acts as a judge and decides whether each response matches the reference. Version 4.1.1 replaced those judges and updated one agentic test, and the firm says most models moved by less than a point. Muse Spark 1.2 rose 2.7 points, the largest increase, and Claude Opus 5 stayed first at 63.

Two changes drove the movement. Humanity's Last Exam (hard expert-level questions), AA-LCR (reasoning across long document sets) and AA-Omniscience (factual knowledge and hallucination) are now all judged by OpenAI's GPT-5.6 Luna at medium effort. It replaces three different judges, GPT-4o, Qwen3 235B and Gemini 3 Flash Preview, and the firm says it chose Luna for strong agreement with human judgment in its grader validation. Separately, τ³-Banking, an evaluation from Sierra in which a model handles a simulated bank customer using tools while following policies, moved to upstream version 1.0.1. The firm says its grader resolves correctness errors in runs where the agent hits a failed step and recovers. Artificial Analysis attributes the small increases to more robust grading: when a judge stops rejecting correct answers or properly credits a recovered run, the score rises with no change in what the model can do.

The practical consequence is that Index scores are comparable only within one version. Muse Spark 1.2 had launched a day earlier at 54; after the patch it sat near 57, and Artificial Analysis's cost-per-task comparison published hours later used the revised scores. A score quoted in July and one quoted after this patch can differ partly because of grading. The firm's methodology history now lists v4.1.1 as in force from August to September 2026; later versions dropped τ³-Banking, and the Index has since been rebuilt as version 5, so any comparison across months needs the version noted beside the number.