Two weeks.
That's how long it took to go from GLM-5.3-Flash's first run on domestic accelerators to serving all of its production traffic, with 3.2× end-to-end throughput along the way.
What I keep thinking about is who did much of the work: an Infra Agent powered by GLM-5.3.
A model helping optimize the system that serves it.
The conditions were hard. Limited memory and interconnect bandwidth. 1M-token context. Multimodal requests. An immature software stack where kernels were missing and documentation was often guesswork. Every optimization was a trade: compute for memory (ReplaySSM), communication for memory (intra-node tensor parallelism), precision for capacity (mixed INT8/FP8/BF16 caching), and disaggregation for scheduling freedom (Encode–Prefill–Decode).
But the most important lesson wasn't about any single optimization.
When the agent got stuck, it was rarely because it couldn't write the code. It was because it didn't know *why* things got worse.
"Throughput down 20%" tells you something broke. It doesn't tell you which layer, which hypothesis, or what to test next. In RL terms, it's a sparse reward with a credit assignment problem. And an end-to-end benchmark…
An Infra AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → powered by GLM-5.3 took GLM-5.3-Flash from its first run on domestic accelerators to serving all production traffic in two weeks, reaching 3.2x end-to-end throughput along the way.
The agent traced a KDA context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →-parallel precision bug to TF32 rounding error compounding through chained state-matrix merges; the fix has been merged upstream into Flash Linear attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → (PR #1180).
It found KV transfer never overlapped with DeepEP dispatch because the intranode path never released Python's GIL; fixing that cut transfer overhead from over 30% to under 1%.
A decode kernel was recomputing the same normalization four times due to how work was chunked; restructuring it gave a 1.71x speedup, an idea drawn from patterns the agent distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → by reading kernels across SGLang, FLA, and DeepGEMM.
Key lesson: the agent rarely got stuck writing code, but on diagnosing why things got worse, so the team built explicit 'dense feedback' (correctness, system-behavior, and performance signals) that are local, cheap, and verifiable.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
Read the full post on X ↗
