Vibeleaderboard
← All Intel
Intel / video

MoE Token Routing Explained: How Mixture of Experts Works (with Code)

Source
youtube.com
Author
Hugging Face
Date

Context

A layer saves computation by sending each input to only some of its component networks. In Hugging Face's routing tutorial, a small example scores four available experts and selects the two highest-scoring choices for each token. The unselected experts receive no assignment for that token.

Selection also has to respect capacity. In this example, each expert has two available slots, so an assignment can be dropped when its selected expert is already full. The walkthrough explains why routing needs both a preference calculation and a capacity rule: deciding where a token should go does not establish that its preferred destination can accept it.

Terms in this piece · Glossary
  • mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
  • token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
  • inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Read the source www.youtube.com
More from Hugging Face
Recommended reads
Comments

Checking sign-in…

Loading comments…