Clip / AI Agents

Common misconception: this is not gradient sparsity

From Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal · ≈12:45

Corrects the intuitive but wrong explanation — density is unchanged, the sparsity appears only in the quantized rollout view.

What’s in it

Corrects the intuitive but wrong explanation — density is unchanged, the sparsity appears only in the quantized rollout view.

Watch this moment

Clip transcript

reconstruct the exact same version. So a common misconception there is like the work it works because all our gradients are sparse. They are not. The paper reports the gradients are dense. About 99% of the parameter gets non-zero gradients. The FP32 master update is also dense. It's just small. The main thing is like the rollout weight change is just 1% from the perspective of rollout engine.