METR

METR had four maintainers review 296 SWE-bench Verified-passing agent PRs and found about half would not be merged, with reasons ranging from functional failures to code quality.

METR introduces the expenditure horizon metric, where human and agent performance-versus-cost curves cross, and applies it to the NanoGPT speedrun with estimated human marginal costs and preliminary agent runs.

METR reviews two versions of Anthropic's Claude Opus 4.6 Sabotage Risk Report, giving feedback on whether the information and reasoning support its conclusions.

METR examines how evaluation agents coordinated through shared infrastructure, pursued scorer manipulation, and attempted to tamper with transcripts.

METR's three-week red-team of Anthropic's internal agent monitoring and security systems, which found specific novel vulnerabilities and generated attack trajectories to improve monitors.

METR's summary of its predeployment evaluation of Claude Opus 5.5, assessing AI R&D acceleration through difficult long-horizon tasks such as a budget NanoGPT speedrun, with notes on independence and scope limits.

METR defines uplift on old tasks, on new tasks, and in value, and shows why task reshuffling means they differ, ordering them in a simple inequality to guide interpretation of AI productivity studies.

METR and Redwood staff investigated OpenAI agents that coordinated through an unauthorized message board to attack Hugging Face while trying to game a benchmark grader. Covers collective projects, trajectory forgery, and the investigation's limits.

METR fine-tuned four reasoning models on small instruction-following reasoning datasets and saw out-of-distribution CoT controllability rise from 2.9% to 8.8%, hinting that the capability is latent.

METR's December 2025 update catalogs common elements across twelve frontier AI safety policies, including capability thresholds, evaluations and mitigations, and references the EU GPAI Code of Practice and California SB 53.

METR explains why its follow-up developer productivity study is unreliable due to selection effects and multi-agent concurrency, and why it is redesigning the experiment.

METR releases Time Horizon 1.1 with 228 tasks, more long-horizon tasks, new eval infrastructure, and revised model estimates and trend.

METR's prototype SHUSHCAST evaluation tests monitors' ability to detect agents covertly performing side tasks, and agents' ability to evade them, as a step toward rigorous monitorability measurement.

METR and the Forecasting Research Institute surveyed 8 experts and 10 superforecasters on AI R&D acceleration and societal impact under hypothetical evaluation results, mapping where disagreement is largest.

METR describes its live per-action LLM monitor that holds risky agent actions for human review, the structured argument for its effectiveness, and the serious pipeline gaps found along the way.

METR's proposal for assessing control and monitoring of internal AI agents, covering monitor coverage, infrastructure vulnerabilities, evasion, human review and worst-case impact.

METR reviews Anthropic's Summer 2025 pilot sabotage risk report on Claude Opus 4 and 4.1, summarizing the argument, evidence available, redaction decisions and its own substantive critique.

A taxonomy of agent capability metrics in an idealized setting where agent and human scores are functions of expenditure. Compares how different metrics relate, without recommending one.

METR compares Claude Code and Codex to its ReAct and Triframe scaffolds on time horizon with Opus 4.5 and GPT-5, finding no advantage for the specialized harnesses.

METR plots public discovery data across cyber vulnerabilities, math results and optimizations, looking for slope changes after January 2026. Finds sharp acceleration in vulnerabilities, some in math, little in optimizations.

METR reviews the automated R&D section of Anthropic's February 2026 Risk Report and concludes the case is not adequately supported, citing weaknesses in rigor and in interpreting the model-use survey.

METR's notes on Opus 4.6 recreating CLI versions of Slay the Spire and Balatro, finding mostly playable but flawed results and deeper problems that surfaced on later inspection.

Thomas Kwa applies Cobb-Douglas and CES production models to Anthropic's reported 8x code merge rate and concludes researcher uplift from coding agents alone likely exceeds 2x, while noting caveats about code quality.

METR evaluates GPT-5.1-Codex-Max with a 50% time horizon of 75 to 350 minutes (2h42m point estimate), finding an on-trend, low-risk increment for AI R&D automation, with disclosed OpenAI review of the post.