vLLM gains an experimental Attention-FFN disaggregation plugin for MoE
- Source
- x.com
- Date
Proud to be part of this collaboration! 🎉 Huge thanks to the vLLM team, Ant Group, and FastAFD. Excited to bring AFD to the open-source community. https://t.co/YvhUL2a0Ea
Context
StepFun's post is a short thank-you crediting the vLLM team, Ant Group, and "FastAFD" for bringing -FFN Disaggregation, or AFD, to the open-source community, but it does not explain what AFD does. That detail comes from vLLM's own plugin announcement, published as an experimental release.
AFD separates a model's attention step, which is mostly limited by memory access speed, from its expert, or feed-forward, step, which is limited by raw compute, letting each run as an independently scaled service instead of sharing one GPU that cannot be tuned well for both at once. vLLM's plugin currently supports DeepSeek's V2 and V3 model family, including V3.2, and GLM's MoE DSA architecture.
- attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
- mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
postStepFun Open-Sources Step Code, a CLI Coding Agent Benchmarked on Terminal-Bench
postStepFun Open-Sources onPanda, Its Internal LLM Annotation and Debug Tool
postStepFun debuts StepAudio 3 voice model family, tops real-time benchmarks- postOpenRouter free tier for Step 3.5 Flash jumps to 10K calls a day
Checking sign-in…
Loading comments…


