Review of the Anthropic Summer 2025 Pilot Sabotage Risk Report
Source
metr.org
Date
Why it matters
An independent reviewer examines Anthropic's argument that sabotage risk from Opus 4 and 4.1 is very low but not negligible. Shows how safety cases are assessed for models used in internal agentic work.
Key takeaways · AI-distilled
METR received unredacted drafts plus additional nonpublic evidence from Anthropic, and judges the public report's redactions justified on balance by stated rationales and accurately marked.
METR's main analytic criticism is of Claim 2, that Opus 4 cannot hide its reasoning on complex tasks: it is imprecise about which tasks are in scope, and METR says it does not cover simple reasoning about misaligned goals or evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → awareness.
METR argues Opus 4 could in theory do significant 'instinctual' misaligned reasoning that the report's evaluations would not reveal, but judges this unlikely given prior models and Opus 4's limited capabilities.
METR still agrees catastrophic sabotage risk from Claude Opus 4 and 4.1 is low, including for models within the report's scope, and recommends incident tracking, additional tests and training data filtering.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
