MoE Token Routing Explained: How Mixture of Experts Works (with Code)
Source
youtube.com
Author
Hugging Face
Date
Context
A mixture-of-expertsA model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.Full definition → layer saves computation by sending each input tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → to only some of its component networks. In Hugging Face's routing tutorial, a small example scores four available experts and selects the two highest-scoring choices for each token. The unselected experts receive no assignment for that token.
Selection also has to respect capacity. In this example, each expert has two available slots, so an assignment can be dropped when its selected expert is already full. The walkthrough explains why routing needs both a preference calculation and a capacity rule: deciding where a token should go does not establish that its preferred destination can accept it.
Terms in this piece · Glossary
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.