Total AI chip memory bandwidth is growing 4.1x per year and is a rough proxy for global capacity to serve models. HBM supply is a key constraint on inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → availability and cost.
Terms in this piece · Glossary
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
