Karpathy's full course from micrograd to nanoGPT, built from scratch in Python. The canonical path from "I use models" to "I understand them." Long, but nothing else replaces deriving backprop yourself.
Key quotes
“A course on neural networks that starts all the way at the basics. The course is a series of YouTube videos where we code and train neural networks together.”
“We implement a bigram character-level language model, which we will further complexify in followup videos into a modern Transformer language model, like GPT.”
“We learn why training deep neural nets can be fragile and introduce the first modern innovation that made doing so much easier: Batch Normalization.”
