Intro to Mixture of Experts | Aritra Roy Gosthipaty | HF Podcast #2
Source
youtube.com
Author
Hugging Face
Date
Why it matters
Gives a practitioner-level intro to mixture-of-expertsA model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.Full definition → architectures, which underpin many current open and frontier models.
Terms in this piece · Glossary
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.