Intel / article

Interconnects: Frontier Post-Training Recipe Review

Source: interconnects.ai
Author: Nathan Lambert
Date: Published Jun 16, 2026

Context

Several specialist models can teach one model to handle a broader range of tasks. In Nathan Lambert's post-training recipe discussion, specialists first develop skills in domains such as mathematics or coding. A general model then learns from the relevant specialists' predictions.

In the on-policy version, the learner generates its own attempts, and a teacher supplies guidance at each next-text decision. Training therefore addresses situations the learner actually produces. The discussion presents this as a way to consolidate separately developed skills into one deployed model, while leaving open how much this approach improves on simply training from teacher-written examples.

Key takeaways · AI-distilled

Multi-teacher on-policy works like this: train several domain-specialist teachers, then have one general student generate its own rollouts and match, by token, the output distribution of whichever teacher owns that domain.
It exists because one big RL run mixing math, code, and agentic tasks eventually trades those capabilities against each other. Specialists trained separately are cheap, well-understood, and do not fight.
The scale-up is fast: MiMo Flash v2 used about six teachers, DeepSeek V4 and Nemotron 3 Ultra push past ten, and Nemotron runs the distillation twice, re-distilling from refreshed teachers.
Llama 3 used its reward model only as a filter, never for online RL: sample K completions per prompt, reject the bad ones, SFT, DPO, repeat six rounds with the best models seeding the next.
Modern post-training is bounded by the org chart as much as by compute. OLMo 3's recipe stayed simple partly because coordinating more stages was already at the limit of what the organization could wrangle.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Key quotes

“The shape of a post-training recipe has changed more in the last year than in the prior three.”

“post-training OLMO-3 to make this reasoning model was a major accomplishment for many individuals to do this. But also, the complexity of what we were doing was pushing against the limits of AI2\”

“One key finding from our trials of doing on policy, multi-teacher on policy distillation is that teacher models trained with substantially different training pipelines cannot be effectively combined through a straightforward on policy distillation merge, resulting in suboptimal performance.”
Nathan Lambert

Read the source www.interconnects.ai