QuasiMoTTo Cuts Wasted Compute in Parallel Inference Sampling
- Source
- x.com
- Date
We love scaling inference compute, but it’s costly! Independently sampling parallel attempts might be the culprit: it wastes compute rediscovering the same solutions. What if we scaled inference compute with correlated samples? Check out QuasiMoTTo by @michaelyli_ and @probablynotaz9!
Context
Generating several independent attempts at a problem and picking the best one can waste compute, because independent attempts can accidentally converge on the same solution rather than exploring different ones. Stanford SAIL's post points to QuasiMoTTo, by Michael Y. Li and coauthors, which reframes sampling as picking points along an underlying number line and spreads those points out evenly instead of scattering them randomly, producing attempts that are correlated but still individually valid.
The authors report this evenly spread sampling reaches the same performance using 25 to 47 percent fewer samples in test-time scaling, and 50 percent fewer training steps in a reinforcement learning setting. These are the paper's own reported efficiency gains under its tested configurations.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Checking sign-in…
Loading comments…