Had a 💥blast building this with the SGLang team, the collaboration shows what scenario-specific optimization open-source can do.
The Humming + DSpark optimizations are already upstreamed. 📖 Full breakdown: https://t.co/QApu2aei3F https://t.co/1Y08FkJyBi
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Why it matters
A 1.6T mixture-of-expertsA model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.Full definition → reaching 271 output tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition →/s at batch size 1 on H20 hardware without native FP4 tensor cores narrows the gap to B300 to 1.42x, and the Humming and DSpark optimizations behind it are upstreamed in SGLang.
