LLaDA2.1 brings token editing to diffusion LLMs at 892 tokens per second
- Source
- x.com
- Date
What if an LLM could EDIT its own tokens in real-time, not just generate them? 🤯 Introducing LLaDA2.1 — a diffusion model that breaks from autoregressive dominance. It drafts fast, then fixes its own mistakes on the fly with Token-to-Token editing. The result? 892 tokens/sec on a 100B model. 🔥 ⚡ 892 TPS on HumanEval+ (coding) ⚡ 801 TPS on BigCodeBench 🧠 Real-time self-correction via T2T editing ✅ @lmsysorg SGLang Day 0 support — production-ready now A "non-consensus" architecture now challenging the mainstream. Open-sourced TODAY. 👇 #LLaDA #TokenEditing #OpenSource #LLM #dLLM
What's the secret? Our revolutionary Error-Correcting Editable (ECE) engine. 🧠 LLaDA2.1 doesn't just write; it drafts, then intelligently reviews and edits its own work, much like a human expert. This two-phase approach fixes a major weakness of early diffusion models — exposure bias and error accumulation.
Flexibility is key. LLaDA2.1 introduces a dual-mode design: ⚡ Speedy Mode for rapid iteration and prototyping 💎 Quality Mode for precision-critical production tasks You choose between speed and perfection based on your needs. From research to real-world applications.
We've also pioneered the first-ever large-scale Reinforcement Learning (RL) framework for a 100B-parameter diffusion model. Through EBPO (Block-level Policy Optimization), LLaDA2.1 learns to understand and follow complex human instructions with incredible fidelity. It doesn't just generate, it understands. 🤝
Context
Most language models commit to each as soon as they generate it, so an early mistake propagates through everything written after it. Ant Open Source built LLaDA2.1 to avoid that by pairing block diffusion generation with what it calls an error-correcting editable engine: the model drafts a block of text, then reviews and edits its own draft before committing to it, rather than only ever appending new tokens. Ant reports the 100B-parameter version reaching 892 tokens per second on HumanEval+ and 801 on BigCodeBench, with same-day support in the SGLang serving engine, and released both a 16B and a 100B version.
The release builds on the company's earlier LLaDA2.0 diffusion model and describes what it calls the first large-scale reinforcement-learning framework applied to a 100B-parameter diffusion model, using a method it names EBPO to train instruction-following.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Checking sign-in…
Loading comments…