Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal

Modal's Stitch approach aims to keep distant AI training and response-generation machines synchronized without repeatedly transferring an entire model. The talk's published description explains that training updates can change high-precision parameters while leaving their rounded, lower-precision serving values unchanged. In the described setup, only a small fraction of the served values therefore need replacing between versions. Stitch sends those changes so the receiving machines can reconstruct the same serving weights. The proposed benefit is less data crossing regions, making it easier to use available response-generation capacity outside the trainer's cluster while leaving the tightly connected training machinery together.

“A frontier scale checkpoint is around 500 GB, so shipping one to a rollout fleet in another region takes minutes to hours and kills any hope of weight updates landing in seconds.”

“Nan Jiang's claim is that you can send roughly 500 MB instead and have the rollout engine reconstruct a bitwise identical weights version.”

“Fewer than 1% of rollout visible weights actually change between consecutive versions, and the reason is not that gradients are sparse. Gradients are dense, about 99% of parameters get a nonzero gradient and the FP32 master update is dense too. It is just small.”

“an internal run serving GLM 4.7 Air in FP8 saw 0.15% of weights change on the first step and settle near 0.05%”