The 'how many months behind is open source' question has no answer because each side picks its favorite benchmark. The ones that track what people actually buy are agentic coding and computer use; the rest measures long-tail robustness.
Kimi K3 is large enough that fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → it takes a full node of B300s just to load the weights, so the community post-training wave that followed Qwen and GLM will need serious engineering before it shows up here.
A big scale-up usually ships with rough-edged post-training, which is exactly why fine-tuning a fresh giant open model pays off: there is performance left in it that the lab did not have time to extract.
Speed beats marginal quality in daily use. GLM 5.2 on a fast endpoint at 200-300 tokens per second wins grunt work because picking a model and a reasoning effort in Codex costs more time than the quality gap buys back.
One concrete finding from their own data: Reddit discussion of a new open model leads its Hugging Face download takeoff by one to two months, making forum chatter an early indicator of which models get adopted.
Terms in this piece · Glossary
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
A groundingTying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.Full definition → roundup of why Chinese open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → models have closed the gap on frontier labs, with practical takes on distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → and the shifting economics of open vs. closed models — useful for engineers deciding which open models to build on.
Key quotes
“So I would say that Kimi K3 is like 54-55 level for these kind of tasks.”
Florian Brand
“And I think this is actually one of the best arguments for not doing anything.”
Nathan Lambert
“So they had to use GLM, a lesser capable model, but it had no guardrails for this kind of defensive action.”
Florian Brand
“And so I just think the argument that distillation is helping more because RL is becoming more prevalent is not grounded in literature that we have today.”
