DFlash
github.com/z-lab/dflash- Category
- AI Tools
- Rank
- No. 1577Tools index
Previous survey · No. 1526 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- z-lab
- GitHub
- 6.1k stars
- Latest release
- v0.1.0
- Date
About
DFlash is a lightweight block diffusion model used as the draft stage in speculative decoding for LLMs. Instead of drafting candidate tokens one at a time, it generates a block in parallel for the target model to verify, cutting inference latency. Pre-trained draft models are published for several model families, and it plugs into vLLM, SGLang, Transformers, and MLX on Apple Silicon.
What it does
DFlash pairs a small drafting model with a larger target model. The drafter proposes a block of continuation tokens from target hidden states, then the target checks that block and accepts the matching prefix before generation continues.
Why it's ranked here
The project is compelling because it connects one drafting method to several practical inference stacks and supplies matching model weights. Its usefulness is narrower than that breadth suggests: backend support differs by model family, and some serving paths still require temporary upstream builds.
What's good
Users can choose among vLLM, SGLang, Transformers, and MLX instead of adopting a single serving stack. The included benchmark runner uses the same five datasets across backends and reports throughput, acceptance length, and acceptance distributions. Draft weights cover several Qwen, Gemma, MiniMax, Kimi, Llama, and GPT-OSS variants.
Tradeoffs
Transformers works only with Qwen3 and Llama 3.1 models. Gemma support under vLLM needs a temporary container or source build, while newer sliding-window drafts need another development branch. The documentation also labels schedule overlapping as experimental and potentially unstable. Separate virtual environments are recommended because backend dependencies can conflict.
How to use it well
Use DFlash when you already serve a supported target model and want to test speculative decoding against your own prompts and hardware. Start with its shared benchmark suite, compare ordinary decoding with block drafting, then tune the block size and backend. It does not provide the promised training recipe for creating new draft models yet.
Technical notes+
pyproject.toml defines a Python 3.10+ setuptools package with separate transformers, sglang, vllm, and mlx extras. dflash/model.py implements dflash_generate, which drafts blocks, verifies them with the target model, crops both caches after partial acceptance, and optionally returns timing and acceptance statistics. dflash/model_mlx.py provides an MLX implementation, downloads draft configurations and safetensor weights from Hugging Face, binds target embeddings and output heads, and handles full or sliding attention caches. dflash/benchmark.py exposes backend-specific benchmark paths, downloads five datasets into a JSONL cache, supports distributed Transformers runs, and calls HTTP endpoints for SGLang and vLLM. dflash/__init__.py lazily exposes the draft model and selected utilities.
Observed
- License
- MIT License
- Primary language
- Python
- Python requirement
- Python 3.10 or newer
- Packaging
- Setuptools package with optional extras for Transformers, SGLang, vLLM, and MLX
- Interfaces
- Python library, module-based benchmark CLI, and integrations with vLLM and SGLang servers
- Platform support
- CUDA-oriented Transformers and serving workflows, plus MLX support for Apple Silicon
- Benchmark coverage
- Shared runner for GSM8K, MATH-500, HumanEval, MBPP, and MT-Bench
Read from README.md, pyproject.toml, dflash/model.py, dflash/__init__.py, dflash/benchmark.py, dflash/model_mlx.py, LICENSE.
Intel on DFlash
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.