If you self-host inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition →, this changes throughput and memory math for DeepSeek V4 and Kimi-K3 serving, and moves Model Runner V2 features like E/P/D disaggregation and weight offloading into reach.
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
