distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
Why it matters
A structured deep-dive into how curriculum learning accelerates and stabilizes RL training — covering task ordering, procedural content generation, and distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition →-based curricula — useful for anyone designing training pipelines for agents or fine-tuned models.
Key quotes
“Back in 1993, Jeffrey Elman has proposed the idea of training neural networks with a curriculum. His early work on learning simple language grammar demonstrated the importance of such a strategy: starting with a restricted set of simple data and gradually increasing the complexity of training samples; otherwise the model was not able to learn at all.”
Lilian Weng
“To design an efficient and effective curriculum is not easy. Keep in mind that, a bad curriculum may even hamper learning.”
Lilian Weng
“They noticed that combined strategy always outperformed the naive curriculum and would generally (but not always) outperform the mix strategy — indicating that it is quite important to mix in easy tasks during training to avoid forgetting”
Lilian Weng
“Interestingly, both works above (in the discrete task space) found that uniformly sampling from all tasks is a surprisingly strong benchmark.”
Lilian Weng
“Alice should learn to push Bob out of his comfort zone, but not give him impossible tasks.”
