The 3-point gain over Muse Spark 1.1 on the Artificial Analysis Intelligence…
- Source
- x.com
- Date
The 3-point gain over Muse Spark 1.1 on the Artificial Analysis Intelligence Index is concentrated in agentic evaluations: GDPval-AA v2 +260 Elo (1371 to 1631), Terminal-Bench 2.1 +2 points (78% to 80%), and Tau3-Bench Banking +2 points (25% to 27%). The minor regressions are SciCode (-2 points) and Humanity's Last Exam (-1 point)
Context
Artificial Analysis found that Meta's Muse Spark 1.2 improved unevenly across its tests. Its August 5 breakdown says the three-point index gain is concentrated in agentic evaluations: GDPval-AA v2 rose by 260 Elo points, while Terminal-Bench 2.1 and the banking each gained two percentage points.
Two benchmarks moved the other way: SciCode declined by two percentage points and Humanity's Last Exam by one, which the post calls minor regressions. Those changes use different scoring scales and should not be added together. They show why a higher overall index can coexist with small declines on particular tests.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Checking sign-in…
Loading comments…