inference

Training builds the model; inference is every request served afterward. When you chat with Claude, that's inference.

It's also where the economics live. Training cost is paid once; inference cost is paid on every single request, forever. Most of the clever systems work in this index — KV caches, batching, speculative decoding, quantization — exists to make inference faster and cheaper, because at scale those pennies per request decide what's viable to build.