Llama 3.2 1B Qwen3.8 27B Draft released
- Source
- huggingface.co
- Author
- NVIDIA model releases
- Date

Context
NVIDIA's Llama 3.2 1B draft is an experiment in reusing a small model from another family to speed up Qwen's Qwen3.8-27B. It is meant for : the small draft guesses several upcoming , the large model checks them all in one pass and keeps the ones that pass a statistical acceptance test, so the final text is distributed exactly as Qwen3.8-27B alone would produce it. The benefit is fewer slow passes through the large model.
A draft has to propose tokens the target can check, so the two must share a vocabulary. Meta's Llama 3.2 1B was built with a different one, so NVIDIA rebuilt its input and output layers on Qwen3.8-27B's vocabulary of 248,320 tokens. That is why the checkpoint has 1.48 billion parameters, more than the original model. NVIDIA then trained it on 199,998 prompts answered by Qwen3.8-27B so it would match the target's choices. The card calls the result a reference point for cross-tokenizer transfer, meaning it shows how well a draft carries over from one model family's vocabulary to another's.
On NVIDIA's own measurements, it carries over less well than the alternatives. Acceptance rate counts the tokens returned for each pass of the large model. On SPEED-Bench, with blocks of 16 guesses, this draft averages 3.42 on the general subset and 3.07 on long prompts, below the 3.81 and 3.86 of the multi-token prediction head built into Qwen3.8-27B, and lower in every category the card lists. The Qwen3.5-4B draft trained the same way averages 5.58. The card reports no timing measurements, but on acceptance alone it gives no reason to choose this draft over the built-in head for serving Qwen3.8-27B.
- speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
Checking sign-in…
Loading comments…



