Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026)

METR agrees with Anthropic that its models posed low risk from automated research as of July 15, 2026, but thinks it is plausible they had already crossed the early-warning threshold Anthropic says they had not. Anthropic's Responsible Scaling Policy, its rulebook for when model capabilities call for stronger safeguards, has two tests here. One is a model that could fully substitute for all of Anthropic's research scientists and engineers at no more than five times the cost. The other is a doubling of the pace of AI progress that is plausibly due to AI automating research. In a published review, METR examined the automated R&D section of Anthropic's August 2026 Risk Report with access to the unredacted private version, and found the substitution argument broadly compelling but the case against a doubling overstated.

The disagreement comes from comparing the public and private wording. The public report says leading indicators show acceleration "by less than a factor of 2". METR calculates that the private report's own figures imply 1.97, which it says a reader would not expect from that phrase, and its separate analysis found a range of 1.3 to 2.8. The private report also called it plausible that the speedup attributable to AI exceeded a factor of two, while the public version, in METR's reading, implied more confidence that only a smaller share came from AI. On benchmark scores, the public report left out Anthropic's private expectation that later versions of an unreleased model, called Model 2, would score higher. They did, by about as much as Claude Mythos 5 had improved on Mythos Preview. METR proposes replacement wording for each claim.

METR still calls the risk low because it thinks catastrophic scenarios would likely require much more than a doubling, and because models at the time could not fully automate research. Anthropic itself describes the doubling threshold as an early warning, not evidence that harm has arrived. METR adds that it cannot rule out a substantial further acceleration over the next six months, which could quickly raise the risk to high. Its companion assessment puts Anthropic's rate of progress at roughly twice the historical pace as of October. In its earlier review of the February 2026 report, METR also agreed with Anthropic's bottom line on low risk but said the evidence in the report did not adequately support it.