Ling-3.0-flash gets an open DSpark speculative decoding draft model
- Source
- x.com
- Date
Today we are open sourcing Ling-3.0-flash-dspark, a DSpark draft model built specifically for Ling-3.0-flash. On 4 NVIDIA Blackwell GPUs at batch 1, it delivered 1,120 tok/s, 0.78 ms mean TPOT, and an accept length of 9.95 across 1,000 requests. 🧵

Context
speeds up a by letting a much smaller draft model guess several upcoming , which the large model then checks in a single pass, keeping the guesses it agrees with plus one token of its own. Because the large model verifies every kept token, its output does not change, and the speed gain depends on how many tokens are accepted per check, a number called the accept length. Ant Ling open-sourced Ling-3.0-flash-dspark, a 1.36-billion-parameter draft model built for its Ling-3.0-flash model. Its model card describes DSpark as an extension of the DFlash drafting method that reads internal features from the target model and adds a confidence head that decides how many tokens to draft at each step. According to the accompanying technical writeup by Ant's Ling infrastructure team and the RadixArk SGLang team, the draft was trained mainly on Ling-3.0-flash's own post-training data and added a loss term that rewards longer accepted runs.
Ant's headline result, on four NVIDIA Blackwell GPUs answering one request at a time across 1,000 requests, is 1,120 tokens per second, 0.78 milliseconds mean time per output token, and an accept length of 9.95. The writeup compares that with a tuned NEXTN setup, the model's built-in path in which a one-layer draft proposes six tokens: 606 tokens per second, 1.53 milliseconds and an accept length of 3.25 on the same . That run used greedy decoding on synthetic random prompts with 8,192 input and 1,024 output tokens, and the writeup says accept length depends on the prompt and output distribution. It also says the 9.95 came from a draft with a block size of 16, while the released checkpoint uses 8. The model card measures the released draft on nine real benchmarks and reports accept lengths from 3.51 on Alpaca to 6.57 on HumanEval, averaging 5.29, with math and code higher than chat. With fewer tokens committed per check, the speedup on ordinary prompts is likely smaller than the roughly 1.9 times shown in the headline comparison, though neither source measures speed on those workloads. A separate Intel item covers the full writeup.
- speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Checking sign-in…
Loading comments…





