Hugging Face Journal Club: Direct On-Policy Distillation

Describes a cheaper route to RL-grade improvements on a large model by reusing the policy shift from a small RL-trained model as a dense reward.

• distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.

• videoHow AI agents reproduced ICML 2026 papers
• videoHow to Run Local Models in Pi
• videoTraining Agents 4: From reward functions to environments.
• videoStructured Output in the browser with Transformers.js 4.3

• postOn-policy distillation: RL's error correction at SFT pricesThinking Machines
• articleOn Policy DistillationThinking Machines
• articleDirect Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov et al.

Checking sign-in…

Loading comments…