Flow-based Deep Generative Models

A normalizing flow builds a complicated probability distribution from a simple one using transformations that can be reversed. Lilian Weng's 2018 tutorial explains why reversibility matters: the model can trace a data point back to the simple starting distribution and calculate its density, accounting for how each transformation stretches or compresses space.

Running the transformations toward the data produces samples; following them backward helps score observations. For this to be practical, both the inverse transformations and their volume-change calculations must be manageable. The resulting density is exact for the constructed model, while how well that model represents real data remains a separate question.

“Neither of them explicitly learns the probability density function of real data”