Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models
Source
Brian K Chen, Chong Wu, Kenji Kawaguchi
Author
Brian K Chen, Chong Wu, Kenji Kawaguchi
Date
Key takeaways · AI-distilled
Draft-then-refine is a plug-and-play decoding pattern for diffusion LMs: generate a complete block-autoregressive draft, then let bidirectional diffusion revise the whole response.
Same-model refinement (Flash-Flash) lifts GSM8K from 0.848 to 0.899 while running 1.20x FASTER than the tuned block-autoregressive baseline, and MBPP from 0.545 to 0.693.
Speculative correction - a small Mini model drafts, Flash edits the draft as an initialization - trades quality for speed usefully: 0.294 vs 0.300 on MATH at 2.17x faster.
Ablations show completed drafts are what matter: refining from a fully masked span performs poorly, while global refinement adds most on GSM8K and local refinement captures most of the gain on MBPP and MATH.
Terms in this piece · Glossary
model routing — Sending each request to a model chosen by the difficulty of the task, rather than using one model for everything.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
Draft-then-refine decoding (Flash-Flash / Mini-Flash) offers a way to speed up diffusion LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition →inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → while improving accuracy without retraining, giving practitioners a new decoding primitive to evaluate for latency-sensitive LLM deployments.
Key quotes
“Flash-Flash improves GSM8K-384 accuracy from 0.848 to 0.899 while running 1.20 times faster than the selected Flash block-autoregressive baseline, and improves MBPP-384 from 0.545 to 0.693.”
“Mini-Flash provides useful quality-latency trade-offs, including MATH-384 performance of 0.294 versus 0.300 for Flash while running 2.17 times faster.”
