context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Why it matters
You can serve Hy4-preview on vLLM today without waiting for a port, and the disclosed layout of 2048 attended tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → per query with sparse index reuse across 57 of 78 layers explains how the 1M window stays affordable.
articleOne GPU, 1M context, Day-0 ready. Big props to the vLLM team for the seamless inAlibaba_Qwen
articleRecent Developments in LLM Architectures: KV Sharing, mHC, and Compressed AttentionSebastian Raschka, PhD
articleEfficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon et al.