What are Diffusion Models?

Diffusion models learn to generate images by learning how to undo noise. Lilian Weng's technical explanation starts with a training image and progressively adds random noise until its recognizable structure disappears. A model learns the reverse transitions: how a less noisy sample could follow from a noisier one.

Generation begins with random noise and repeatedly applies that learned reverse process. The training images supply examples of the structure the model should recover as noise is removed. This connects a seemingly destructive training procedure to image creation: deliberately corrupting data supplies the learning problem used to build the generator.

“GAN models are known for potentially unstable training and less diversity in generation due to their adversarial training nature. VAE relies on a surrogate loss. Flow models have to use specialized architectures to construct reversible transform.”

Lilian Weng

“Diffusion models are inspired by non-equilibrium thermodynamics. They define a Markov chain of diffusion steps to slowly add random noise to data and then learn to reverse the diffusion process to construct desired data samples from the noise.”

Lilian Weng

“Unlike VAE or flow models, diffusion models are learned with a fixed procedure and the latent variable has high dimensionality (same as the original data).”

Lilian Weng

“Compared to standard SGD, stochastic gradient Langevin dynamics injects Gaussian noise into the parameter updates to avoid collapses into local minima.”

Lilian Weng