• KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
• context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Network topology, not GPU count, was the binding constraint on long-context serving here, and the fix was architectural: same GPUs and software, measurably lower tail latency and capex.

• postGarbled tokens were a KV cache race, not a model problem
• postGLM-Image pairs autoregressive planning with diffusion
• postGLM-5.3 arrives via Coding Plan, API and weights staged
• postGLM-5.1 pushes open weights toward eight-hour autonomous runs

• videoKV Cache-Aware Routing and P/D Disaggregation on Kubernetes — Yuchen Fama & Ashish Kamra, Red HatAI Engineer
• articleAnother Giant Leap: The Rubin CPX Specialized Accelerator & RackDylan Patel
• postOff-chip bandwidth has always been the weak point of wafer-scale.SemiAnalysis

Checking sign-in…

Loading comments…