• attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
• mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

MoE serving wastes capacity because attention and expert paths have very different resource profiles. This plugin splits them, giving vLLM operators a supported path to test disaggregated MoE inference.

• postOpenRouter free tier for Step 3.5 Flash jumps to 10K calls a day
• article👏🏻Congratulations！Step3-VL-10B was selected for HuggingFace Daily Papers…
• postStep-Audio-R1.1 opens weights for a speech model that reasons in real time
• postNextStep-1.1 refines StepFun's image model with flow-based RL

• articleGlobal-batch load balance almost free lunch to improve your MoE LLM trainingQwen Team
• postInside the vLLM Changes That Multiplied Agentic Inference ThroughputSemiAnalysis_
• articleEfficient Memory Management for Large Language Model Serving with PagedAttentionWoosuk Kwon et al.

Checking sign-in…

Loading comments…