How Much Does It Cost To Train Frontier Ai Models

This is the summary page for a 2024 paper by Ben Cottier, Robi Rahman, Loredana Fattorini, Nestor Maslej and David Owen of Epoch AI and Stanford's AI Index, with a January 2025 correction that left the headline growth rates unchanged (epoch.ai). Its main estimate is that the amortized hardware and energy cost of the final training run for frontier models has grown about 2.4x per year since 2016. "Frontier" here means models that were among the top 10 by training compute when released, 45 of them in all. Amortized cost charges each run only for the share of the hardware's value used up during training, plus the electricity, rather than the full purchase price of the cluster.

The authors check that figure with a second method: pricing the same runs at cloud rental rates. That gives a similar growth rate of 2.6x per year, but costs around twice as high on average, which the authors attribute to many frontier models being trained on hardware the developer owns rather than rents. If the trend continues, they say, the largest training runs will cost more than a billion dollars by 2027, leaving frontier training to only the best-funded organizations.

A third approach looks at the whole cost of developing a few models, OpenAI's GPT-3 and GPT-4, Meta's OPT-175B and Google's Gemini Ultra, including compute for experiments and R&D staff pay. Hardware makes up 47-67% of that total and staff 29-49%, including equity, with energy only 2-6%. Energy is cheap relative to the rest, but the power demand is not: Gemini Ultra is estimated to have needed 35 megawatts, and a naive extrapolation points to gigawatt-scale supply for AI supercomputers by 2029. Epoch's later estimate of OpenAI's 2024 compute spending fits the point that a final training run is only part of the bill: it found most of OpenAI's R&D compute that year likely went to research and experimental runs rather than the final training runs of released models.