as a whole. So the next obvious move is to add some kind of global signal, something that looks at the draft more holistically to identify inconsistencies and guide token re-masking. A reasonable way to do this is guided diffusion proposed by Flash DML, a lightweight auto-regressive model that supervises the unmasking process. Given an intermediate draft proposed by the diffusion model, the supervisor makes next-word predictions strictly for the recently unmasked positions. The tokens in the draft are then compared against the supervisor predictions. In our case, for the second mask, the supervisor assigns near-zero probability to Madrid, which is already in its left context. Since this is very inconsistent with a sharp distribution around Madrid generated by the diffusion model, this token is re-masked. At first, guided diffusion might seem counterproductive for latency. We're reintroducing an auto-regressive model, so how can we possibly beat an auto-regressive baseline? The reason is the supervisor isn't a fully-fledged generator, but rather a verifier. Because the diffusion model provides a full draft upfront, the ARM can process the entire sequence in a single forward pass and make predictions for the masked positions simultaneously. For instance, Lisbon can be predicted straight away based on this prefix from the draft. And so can Rome based on this other readily available prefix. The auto-regressive supervisor adds minimal overhead. In fact, Flash DML was 12 times faster than LaMDA.