Epoch AI

Epoch AI audits SWE-bench Verified: error rates are low, but tasks are mostly simple fixes in few Python repos, many predate 2020, and scaffold design moves scores as much as model choice. A guide to interpreting coding-agent leaderboards.

Epoch's Furniture Assembly Benchmark tests whether models find mistakes in photos of partly built IKEA furniture. Top scores climbed from 28% to 80% in ten months, and open-weight models trail closed ones by roughly seven months.

Epoch AI interviewed AI researchers about automating AI R&D. Coding and debugging look most automatable, timelines diverge widely, and the report collects researchers' suggestions for evaluations that measure progress.

Epoch AI measured long-context latency to one million tokens. GPT-5.6 Terra and Sol show quadratic time-to-first-token growth, while Claude Sonnet 5 and Opus 5 stay near-linear. Code and data are public, and the post discusses cost implications.

Epoch AI tracks cumulative HBM shipments across AI chips. Aggregate memory bandwidth has grown 4.1x per year to roughly 70 million TB/s by Q4 2025, and AI chips took over 90% of HBM industry revenue in 2025.

Epoch AI shows aggregate cash capex at Microsoft, Amazon, Alphabet, Meta and Oracle rising faster than operating cash flow, on track to overtake it around Q3 2026, with most already turning to external financing for AI buildout.

Epoch AI analyzes how data movement during distributed training caps LLM scaling at roughly 2e28 FLOP, with a latency wall at 2e31 FLOP, and discusses batch size scaling and other ways around those limits.

Epoch AI finds China's six leading AI firms earn roughly a tenth of OpenAI and Anthropic's combined AI revenue. It examines consumer apps, API access, enterprise and government deals, licensing and indirect monetization, and notes open weights limit pricing power.

Epoch's SpireClimb benchmark shows frontier models jumping to over 90% win rates on Slay the Spire's base difficulty within six months, even via visual computer use, and winning up to 20% at the hardest setting.

Epoch AI tabulates gigawatt-scale AI data center timelines: facilities that broke ground in the past three years take 1 to 3.6 years to reach 1 GW, with xAI projecting 12 months for Colossus 2.

Epoch AI breaks down OpenAI's 2024 cloud compute: about $5B R&D versus about $1.8B inference. Final GPT-4.5 training is estimated near $400M, so most R&D compute went to research and experimental runs.

Epoch's EBR-bench has frontier agents play Earthborne Rangers repeatedly. They show no improvement over 30 playthroughs, execute tactics poorly, underexplore strategy and struggle even with an explicit guide, with elicitation gaps acknowledged.

Epoch AI argues that, given the training-inference compute tradeoff, frontier labs should invest roughly equally in training and serving. It uses token-volume figures to show why the two costs end up on the same order of magnitude.

Epoch AI fits competing trend curves to four capability metrics and compares out-of-sample accuracy. Three of four, including METR's time horizon, show acceleration relative to a linear trend, seemingly driven by reasoning models.

Epoch AI estimates Google is the largest single owner of AI compute with about a quarter of cumulative global capacity as of Q4 2025, driven mainly by custom TPUs rather than NVIDIA GPUs.

Epoch AI sketches a 10 GW decentralized training run spread over about a dozen sites thousands of kilometers apart, assesses throughput and network costs, and argues companies will build multi-gigawatt data centers before turning to decentralized training.

Epoch AI charts monthly high and critical CVEs from 21 notable organizations: disclosures spiked in 2026, with June at roughly 3.5x the pre-Mythos monthly record, coinciding with Claude Mythos Preview and Project Glasswing.

Epoch AI tests lab-reported GPQA Diamond scores for recent top models against its own standardized runs. All reported scores land well within its confidence intervals, so there is no evidence of inflation.

Epoch AI estimates the resources behind Grok 4's training run: roughly $490 million in compute cost, 310 million kWh of energy, 750 million liters of cooling water and 140,000 tons of CO2.

Epoch AI describes how its Capabilities Index links benchmarks of differing difficulty into one scale, addressing saturation, and shows how ECI scores map back to expected performance on individual benchmarks.

Epoch AI argues frontier training runs longer than about nine months are outcompeted by later starts with better hardware and algorithms, and projects runs will hit that ceiling around 2027 (90% CI Aug 2025 to Sept 2029).

Epoch AI breaks down AI chip component spending for Nvidia, AMD, Google and Amazon. Memory rose from 52% to 63% of cost, while logic stayed near 13% and packaging fell to 15%.

Epoch AI finds the Epoch Capabilities Index frontier has risen linearly at 14 points per year since o1-mini and o1-preview launched in September 2024, versus 6 points per year for non-reasoning models.

Epoch's Automation Reports give six models realistic Epoch tasks graded manually. Fable 5.1 and GPT-6 Astra lead, open-weight models lag, and all struggle to design informative experiments and often report artifacts of flawed setups as findings.