• mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
• inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
• streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.

It's a benchmarked demonstration that frontier-scale MoE models can run on consumer hardware via SSD streaming, with a specific diagnosed bottleneck (prefill re-reading expert weights eight times) pointing to where real speedups will come from.

• postRunning Kimi K3 on my desk?Alex Cheema
• articleWhy Kimi K3's Architecture Is a Masterclass in AI EfficiencyAlphaSignalAI
• articleKimi K3: The open-weights escalationNathan Lambert

Checking sign-in…

Loading comments…