Intel / post

AngelSpec opens Tencent's speculative decoding training and serving stack

Source: x.com
Date: Published Jul 29, 2026

Context

Tencent's AngelSpec trains smaller drafting models that propose several text for a larger model to check together. Accepting several proposals in one check can reduce the work needed to generate an answer. Its DFly design drafts a block in parallel, then uses a small sequential component to account for the choices before each position. Technical report.

In the authors' Hy3-A21B serving tests, DFly reached 1.98 to 2.40 times the output throughput of ordinary one-token-at-a-time generation, averaged across six datasets at each tested concurrency from 4 to 64. That range describes averages under simultaneous-request loads, not a uniform improvement for every request. The release supplies the training framework and drafter weights for these methods. Results, Table 7, released framework.

Terms in this piece · Glossary

token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.

Read the source x.com