and short inference ones. One promising way to get the best of both worlds is knowledge distillation. Distillation is an older training technique that can be applied to any kind of deep learning models, including auto-regressive and diffusion LLMs. It normally involves two models, an accurate but expensive teacher that shared its knowledge with a smaller but faster student. For diffusion models in particular, it's very common to do self-distillation. A single model is pre-trained on diffusion paths with a reasonably large size N. Once pre-trained, the weights are duplicated. One copy is frozen and will act as the teacher. The other copy acts as the student. It will be fine-tuned to produce shorter paths, say N over two. The teacher paths provide a supervision signal and the student learns to align every state with every second state in the teacher path. The end result is a model that is now twice as fast and just as good. And we don't need to stop here. We can repeat the distillation process. Duplicate the student, freeze one copy, and fine-tune the other to shrink the paths yet again. Distillation is a post-training technique. It basically fine-tunes an existing diffusion model. But there's ongoing exploration on how to adapt the pre-training process as well, so that lowering the refinement steps is not just an afterthought. One direction is curriculum learning. So, under normal circumstances, at each training step, we randomly pick a state or a noise level and teach the model to find its way back to clean data from there. In other words, all task difficulty levels are uniformly spread across the training process. The philosophy behind curriculum learning is to ease the model into it. Train it on easier tasks first with reasonably clean data. And as the training progresses, so does the difficulty level. A bit like a school curriculum. It turns out this makes the model more robust and efficient, able to achieve more in a single step. And that indirectly leads to fewer refinement steps. Distillation and curriculum learning are