Build nanoGPT

Andrej Karpathy's Build nanoGPT uses the repository's change history as part of the lesson. It begins with an empty file and builds toward a reproduction of the 124M-parameter version of OpenAI's GPT-2 model. Deliberately staged commits let learners inspect how the implementation grows alongside the video explanation.

The endpoint is a model trained to continue text. The README explicitly leaves conversational fine-tuning outside this lesson, so reproducing the model is a different milestone from building a chat assistant. Learners needing the earlier training concepts can start with Neural Networks: Zero to Hero.

“While the GPT-2 (124M) model probably trained for quite some time back in the day (2019, ~5 years ago), today, reproducing it is a matter of ~1hr and ~$10.”

“We basically start from an empty file and work our way to a reproduction of the GPT-2 (124M) model.”

“The git commits were specifically kept step by step and clean so that one can easily walk through the git commit history to see it built slowly.”