Total AI chip memory bandwidth is growing 4.1x per year and is a rough proxy for global capacity to serve models. HBM supply is a key constraint on inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → availability and cost.
Key takeaways · AI-distilled
Epoch AI estimates that AI chips shipped since 2022 had roughly 70 million TB/s of cumulative memory bandwidth by Q4 2025, with a growth trend of about 4.1x per year.
The estimate combines quarterly chip shipments with memory-bandwidth specifications across Nvidia, AMD, Google, Amazon and Huawei; it excludes smaller manufacturers and tracks shipments rather than deployed capacity.
Epoch models the H100-to-H200 shipment mix using a logistic curve constrained by six public data points, propagating uncertainty through Monte Carlo sampling.
Its variant-sensitivity analysis gives a 90% interval of 68–75 million TB/s for aggregate bandwidth and 4.0–4.5x for annual growth; some Huawei and AWS chip specifications rely on public-reporting estimates.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
