Sonnet 4.6 edges Opus 4.6 on agentic knowledge work but used 280M tokens versus 160M, so its run cost is slightly higher. Model choice should account for token cost, not only score.
Key takeaways · AI-distilled
The 1633 Elo came from pre-release testing with adaptive thinking at max effort, settings introduced with Opus 4.6, and implies an expected win rate above 85% against Sonnet 4.5.
tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → use rose from 58M for Sonnet 4.5 with extended thinking to 280M for Sonnet 4.6 with adaptive thinking, more than 4x its predecessor.
Its first place sits within the 95% confidence interval of Opus 4.6, so the two are not separable on this benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition →; both Sonnet 4.6's thinking and non-thinking variants cost slightly more to run than their Opus counterparts.
GDPval-AA builds on OpenAI's September 2025 GDPval dataset of self-contained work tasks across 44 occupations in 9 sectors, with models working through shell and web access in the open-source Stirrup agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition →.
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
