Vibeleaderboard
← All Intel
Intel / article

Independent Assessment of AI R&D Acceleration at Anthropic in October 2026

Source
metr.org
Date

Context

METR, an outside group that evaluates frontier AI systems, concluded after an eleven-week review inside Anthropic that the company is likely improving its models' capabilities on well-specified tasks at about twice the rate it managed between March 2024 and October 2025. Twice the rate means making two years of the earlier progress in one year. METR calls the estimate highly uncertain and says it would not be surprised if the true figure were closer to 1.3 or 3 times. The question matters because, in METR's framing, AI that significantly speeds up the development of further AI could lead to extreme and risky capabilities emerging suddenly (METR's public report).

The main evidence METR could publish comes from AECI, Anthropic's version of the Epoch Capabilities Index, which combines many scores into a single capability measure. Counting Anthropic's unreleased internal models, AECI has grown since October 2025 at 1.9 times its earlier rate, with a 95% confidence interval of 1.5 to 2.8. That is higher than the 1.7 times reported in Anthropic's Haiku 5.5 system card, mainly because the newer internal models are included. Reasonable alternative analysis choices give anything from 1.1 to 2.9 times, and the assumption about how task difficulty is spread within each benchmark moves the result most. Adding other companies' models lowers the estimate to 1.4 times, because the earlier trend then comes mostly from other labs, whose models had improved faster than Anthropic's before late 2025. METR offers two readings: either Anthropic's earlier models lagged the frontier, which understates its historical pace and makes 1.4 more accurate, or other labs' earlier models were more tuned to benchmarks, which makes 1.9 closer. METR says it is confident the speedup did not come from faster growth in training compute.

METR's guess is that perhaps 80% of the current acceleration comes from Anthropic's use of AI, while other factors likely played a large role in late 2025. Researcher interviews pointed to a wide range of estimates centered on a doubling. Informal Slack polls of staff reached an average self-reported speedup of about 12 times for the latest internal model, which METR treats as only very loose evidence and notes is not the same as an acceleration in overall progress. AI mostly helps as an assistant to researchers, though staff described "signs of life" for fully autonomous contributions. Because METR found no clear bottleneck in algorithms, compute or data over the next 12 months, it sees a significant risk of even faster acceleration, while noting its view of those bottlenecks changed several times during the review.

The public cannot check most of this. Anthropic held unilateral redaction rights and, according to METR, redacted a majority of the evidence METR found most informative, including its detailed estimates for parts of the training pipeline, its breakdown of compute, data and algorithmic progress, and forecasts for future models. METR says the report therefore does not achieve transparency about evidence and readers must rely on its judgment more than would be ideal. Anthropic also dropped a planned check of how it splits compute between capabilities and safety work. The same METR team separately reviewed Anthropic's August 2026 Risk Report, where it argued that the company's public account understated the acceleration already under way by July.

Key takeaways · AI-distilled
  • METR estimates Anthropic's benchmark index (AECI) has grown 1.9x faster since October 2025 than before (95% interval 1.5x to 2.8x), roughly two years of prior progress in one year, and is confident faster training-compute scaling did not cause it.
  • METR guesses about 80% of the October 2026 acceleration is due to AI use, while the late-2025 speedup was likely shaped by non-AI factors; Anthropic redacted most of the evidence METR found most informative.
  • The acceleration estimate depends on the assumed task-difficulty link function, ranging from 1.1x to 2.9x, while varying snapshots, benchmarks or regularization keeps it between 1.9x and 2.2x.
  • Benchmarks are saturating: an internal model METR calls Model 4 scored above 50% on all 15 AECI benchmarks it was tested on, whereas Opus 4.6 scored below 50% on many about nine months earlier.
  • Anthropic staff self-reported about 12x uplift in October 2026 Slack polls, but METR treats this as loose evidence and notes individual speedup does not map directly to a faster rate of capability progress.
Terms in this piece · Glossary
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Recommended reads
Comments

Checking sign-in…

Loading comments…