karpathy

Experiment in which AI agents autonomously research single-GPU nanochat training, proposing hyperparameter tweaks.

Minimal codebase for training and fine-tuning medium-sized GPT models, with a training loop of roughly 300 lines.

Full-stack ChatGPT-style training and inference recipe that runs end to end on a single 8xH100 node.

Karpathy's tiny scalar autograd engine with a PyTorch-style neural net API, built for teaching backpropagation.

Karpathy experiment where an ensemble of LLMs debates a question and arrives at a synthesized answer.

Karpathy's GPT training implementation in plain C and CUDA, kept small and readable as a reference from the metal up.

Runs Llama 2 inference in a single file of pure C, serving as a minimalist reference for how LLM inference works.

Minimal PyTorch re-implementation of GPT in about 300 lines, written for understanding transformer language models.

Minimal, clean implementation of the Byte Pair Encoding tokenization algorithm used by GPT and most modern LLMs.

Python tool from Andrej Karpathy for reading books together with LLMs, annotating and asking questions along the way.

Renders any git repository into a single static HTML page for human reading or pasting into an LLM.

Uses LLMs to revisit decade-old Hacker News discussions and score how their predictions and ideas held up.

Character-level autoregressive language model that generates new names or words from a small training set.

Rust implementation of BPE tokenizer training by Andrej Karpathy, supplying the training code tiktoken lacks.

Simple web page for logging daily calories, built as a personal tool by Andrej Karpathy.