A rigorous walkthrough of why overparameterized deep networks generalize instead of overfitting, covering the Lottery Ticket Hypothesis, intrinsic dimension, and generalization bounds — foundational intuition for anyone reasoning about model capacity and training behavior.
Key quotes
“Since a typical deep neural network has so many parameters and training error can easily be perfect, it should surely suffer from substantial overfitting.”
