Eci Bayesian Methodology

Epoch AI has rebuilt the statistics behind its Epoch Capabilities Index (ECI), a single score that summarizes how capable an AI model is from its results across many benchmarks. The new method leaves frontier models' scores almost where they were, but it gives better-founded uncertainty ranges and, for the first time, the probability that one model is more capable than another.

The index borrows item response theory from educational testing, where it is used to compare students who sat different exams. Each benchmark gets a difficulty and a slope, each model gets one capability number, and a model's predicted score depends on how its capability compares with the benchmark's difficulty. The scale is pinned so that Claude 3.5 Sonnet sits at 130 and GPT-5 at 150. Most models have only a handful of benchmark scores: ten for the median model and as few as four.

That scarcity is why Epoch replaced its old uncertainty method. It used to draw 500 resamples of each model's scores and refit the whole index each time, but with so few scores, each resample left out about a third of them, and the spread said little about real uncertainty. The new Bayesian model estimates every model and benchmark together and produces a probability distribution for each model's score. It also lets each benchmark have its own noise level and keeps predictions between 0 and 100%. On scores it had not been fitted to, its 90% ranges contained 89.6% of results, against 84.2% for the old method.

For the 176 models at 125 or above, which include every frontier model, scores moved by about one point on average and all stayed inside their previous ranges; the 76 weaker models all moved down, by 13.7 points on average. The new comparisons show how close the top is. GPT-6 Astra's 90% range runs from 166.2 to 172.7 and Claude Opus 5.5's from 165.8 to 172.0, and Epoch gives GPT-6 Astra a 72% probability of being the more capable of the two, against 99% over Claude Fable 5.1. A 72% edge is a lean, not a settled ranking.