Anthropic released Claude Opus 4.1 on August 5, 2025 as a coding and reasoning upgrade at Opus 4's price. Its announcement reported 74.5% on SWE-benchThe standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.Full definition → and highlighted customer reports of better changes across multiple code files.
SWE-bench Verified tests software issue resolution using a human-filtered subset of real repository tasks. The reported score therefore concerns completing bounded repairs in an evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → setup. It provides evidence for coding capability, while the customer accounts describe how that capability appeared in development workflows.
Terms in this piece · Glossary
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
