To make the best of both worlds, the industry seems to be converging towards block diffusion, a mixture of diffusion and auto regression. Given a long context window, tokens are grouped into blocks. Within each block, tokens are decoded using diffusion, but the blocks themselves are processed sequentially, one after the other. Compared to vanilla diffusion models, this has two major advantages. First of all, decoding can stop early. Once a block produces the end of sequence token, the response can be finalized without diffusing the subsequent blocks. And second, bringing back auto regression allows us to repurpose most optimizations developed for traditional models, including KV caching or speculative decoding. But this time we're less concerned with memory bandwidth since diffusion keeps us in the compute bound region. With block diffusion, we're finally approaching an architecture that neither wastes memory bandwidth, nor leaves the processor idle. We're now starting to see the first inference engines that specifically target diffusion LLMs. On the open source side, there's xDiT, WeDLM, and dInfer, all coming from Chinese labs. On the US side, Inception built their own proprietary inference engine, which enables them to serve diffusion LLMs at more than a thousand tokens per second. They also recently launched Mercury 2, the first diffusion reasoning LLM, which is 10 times faster than other reasoning models on the market.