Strategic Warning For Ai Risk Progress And Insights From Our Frontier Red Team

Anthropic reported in March 2025 that Claude 3.7 Sonnet solved about a third of the challenges in Cybench, a public benchmark of capture-the-flag exercises (tasks that ask a model to find and exploit flaws in software in a controlled setting), within five attempts. Its frontier model a year earlier solved about 5%. The red-team report treats that progress as an early warning about capabilities that can support defense or attack, while saying present-day models still fall short of the thresholds at which it sees substantially elevated national security risk.

In more realistic exercises on a network of about 50 hosts, the models could not succeed autonomously. Equipped with software tools built by cybersecurity researchers, though, Claude and other models used simple instructions to replicate an attack similar to a known large-scale theft of personal data from a credit reporting agency. Anthropic says this evaluation setup is meant to warn it when autonomous capabilities improve.