Intel / post

Dense Feedback Let an Infra Agent Serve Its Own Model in Two Weeks

Source: x.com
Date: Published Sep 17, 2026

Two weeks. That's how long it took to go from GLM-5.3-Flash's first run on domestic accelerators to serving all of its production traffic, with 3.2× end-to-end throughput along the way. What I keep thinking about is who did much of the work: an Infra Agent powered by GLM-5.3. A model helping optimize the system that serves it. The conditions were hard. Limited memory and interconnect bandwidth. 1M-token context. Multimodal requests. An immature software stack where kernels were missing and documentation was often guesswork. Every optimization was a trade: compute for memory (ReplaySSM), communication for memory (intra-node tensor parallelism), precision for capacity (mixed INT8/FP8/BF16 caching), and disaggregation for scheduling freedom (Encode–Prefill–Decode). But the most important lesson wasn't about any single optimization. When the agent got stuck, it was rarely because it couldn't write the code. It was because it didn't know *why* things got worse. "Throughput down 20%" tells you something broke. It doesn't tell you which layer, which hypothesis, or what to test next. In RL terms, it's a sparse reward with a credit assignment problem. And an end-to-end benchmark…

Sep 17, 2026 View on X

Read the full post on X

Context

Jie Tang writes that Z.ai took two weeks to go from the first run of its GLM-5.3-Flash model on domestic, meaning Chinese-made, accelerators to serving all of that model's production traffic, with end-to-end throughput up 3.2 times along the way. He credits much of the engineering to an Infra , a coding agent running on the larger GLM-5.3, so the model was helping tune the system that serves its smaller sibling. Z.ai's own announcement gives the same story in rounder numbers: production readiness in under two weeks, with throughput tripled against the initial baseline. Both are the company describing its own project. No outside measurement is offered.

Tang's main point is about why the agent got stuck. He says it rarely failed to write code. It failed because a number like "throughput down 20%" says something broke but not which layer, which hypothesis or what to test next, and a full run took hours. His team's fix was dense feedback: verification tools the agent can call directly, at three levels. One checks whether a component computes the right answer, one shows where the time goes, and one compares options under stated conditions. Each check is meant to be local, cheap and objectively verifiable. Senior engineers make the same judgments in their heads, and the change was to make them explicit tools.

The examples show what that bought. The agent traced a precision error in one path to rounding that compounded as sequences grew, and Tang says the fix was merged upstream into the Flash Linear Attention library. It followed a call chain across the boundary between Python and C++ and found a step that never released Python's global interpreter lock, so moving the model's never overlapped with other communication. Tang says that fix cut transfer overhead from over 30% to under 1%. He also says humans defined the goals, built the feedback environment and reviewed every high-risk change. On his account, the engineer's job moves toward designing those checks, and he describes recursive self-improvement as still far off.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.

Read the source x.com