Are Deep Neural Networks Dramatically Overfitted?
lilianweng.github.io- Category
- Other
- Type
- ARTICLE
- Builder
- @lilianweng
- Added
- Jul 21, 2026
About
[Updated on 2019-05-27: add the section on Lottery Ticket Hypothesis.] If you are like me, entering into the field of deep learning with experience in traditional machine learning, you may often ponder over this question: Since a typical deep neural network has so many parameters and training error can easily be perfect, it should surely suffer from substantial overfitting. How could it be ever generalized to out-of-sample data points?
Why it made the leaderboard
A rigorous walkthrough of why overparameterized deep networks generalize instead of overfitting, covering the Lottery Ticket Hypothesis, intrinsic dimension, and generalization bounds — foundational intuition for anyone reasoning about model capacity and training behavior.
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.