Consistency diffusion language models: Up to 14x faster inference without sacrificing quality
Source
Together AI
Author
Together AI
Date
Terms in this piece · Glossary
KV cache — The memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Why it matters
If you're deploying diffusion language models, CDLM removes their two biggest practical blockers — no KV cacheThe memory a model keeps about text it has already read, so generating each new token doesn't require reprocessing the whole conversation.Full definition → and excessive refinement steps — via a post-training recipe that yields up to 14.5x lower latency without quality loss, making diffusion LMs viable for real inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → workloads.
Key quotes
“We introduce consistency diffusion language models (CDLM), which accelerates diffusion language model inference by combining consistency-based multi-token finalization with block-wise KV caching, achieving up to 14.5x latency speedups on math and coding tasks.”
Together AI
“Standard DLMs commonly use bidirectional (non-causal) attention, which requires recomputing attention over the full context at every denoising step, making inference expensive and preventing standard KV caching.”
Together AI
“CDLM–Dream achieves the largest step reductions across benchmarks, cutting refinement steps by roughly 4.1x–7.7x with minor accuracy changes on most tasks.”
Together AI
“Naively truncating the number of steps causes marked accuracy degradation, while CDLM maintains quality at similar step budgets (and achieves roughly half the latency thanks to caching). This highlights the core point: stable multi-token refinement is not free; it requires training that enforces trajectory-consistent behavior.”
Together AI
“Vanilla DLMs are compute-bound even at bs=1 because full bidirectional attention processes the whole sequence each step, leading to saturation.”
videoMaking Diffusion LLMs Faster: A Survey of Speedup TechniquesJulia Turc
articleBlock Diffusion Interpolating Between Autoregressive And Diffusion Language Models 2025 03 12Cohere
articleSpeculative Correction: Draft-then-Refine Decoding for Diffusion Language ModelsBrian K Chen, Chong Wu, Kenji Kawaguchi