• Explains how Moonshot AI keeps a 2.8T-parameter model deployable via LatentMoE compression, a KV-cache-capping hybrid attention scheme, and aggressive quantization down to community 1-2 bit GGUF — concrete architectural patterns engineers building large MoE systems can study and reuse.
• An analysis of Moonshot AI's Kimi K3, a 2.8-trillion-parameter multimodal MoE model with only ~104B active parameters per token and a 1M-token context window.
• It details the engineering that makes such scale deployable: LatentMoE compression for expert routing, a hybrid Kimi Delta/Gated Latent Attention scheme that caps KV-cache growth, and FP4/FP8 quantization-aware training (with community 1-2 bit GGUF compression) that shrinks the model far below its raw BF16 footprint while preserving reasoning quality.

• articleClaude Opus 5 Debugging Benchmark: Does More Reasoning Actually Fix More Bugs?
• articleThe Open-Model Race has Split Four Ways
• articleModel Routing Layers and the Prompt Cache Tax
• articleGemini 3.6 Flash Debugging Benchmark: It Can Fix Real Bugs, but Is It Efficient?

• articleKimi K3: The open-weights escalationNathan Lambert
• articleKimi K3: The Manos, The Mythos, The LegendosKimbo Chen
• articlemoonshotai/Kimi-K3simonwillison.net

Sign in to comment.

Loading comments…