A dense model uses all its parameters on every token. A mixture-of-experts model contains many "experts" and a router that picks perhaps 2 of 64 for each token — so a model can have a trillion parameters total but only run a fraction of them per step.

That's the trick behind many frontier and open-weight models: capability tends to scale with total parameters while cost scales with active ones. The trade: the whole thing must still sit in memory, and routing adds its own engineering (load balancing, training stability).