Strategic Warning For Ai Risk Progress And Insights From Our Frontier Red Team
Source
anthropic.com
Author
Anthropic research
Date
Why it matters
Shows measured jumps in model cyber capability (Cybench solve rate from about 5% to about a third in a year) and how a lab designs evals to avoid training-data contamination. Useful for anyone building or securing agents.
Key takeaways · AI-distilled
Anthropic reports that Claude still lags expert humans at reverse engineering binaries to find hidden vulnerabilities and at reconnaissance and exploitation inside a network, at least without some help.
On roughly 50-host cyber ranges built with Carnegie Mellon, models could not succeed autonomously, but with a researcher-built toolkit Claude and other LLMs replicated an attack similar to a known large-scale theft of personal data from a credit reporting agency.
In biology, Anthropic says Claude went from underperforming world-class virologists on a lab troubleshooting evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → to comfortably exceeding them within a year; its latest model beats expert baselines on cloning workflows but still trails at interpreting scientific figures.
In a controlled study, Anthropic found its newest model gave novices some uplift on weaponization-related tasks, but even the best model-assisted plan still contained critical mistakes that would cause failure in the real world.
Claude went through pre-deployment testing by the US and UK AI safety institutes, and the NNSA evaluated it in a classified environment for nuclear and radiological knowledge, with government staff doing that red-teaming directly.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
