karpathy

Andrej Karpathy's experiment: AI agents that autonomously run research on single-GPU nanochat training, proposing hyperparameter tweaks and summarizing findings.

A minimal, fast implementation for training and fine-tuning medium-sized GPT models from scratch. Features clean, readable code with just ~300 lines each for the training loop and model definition, making it easy to customize and experiment with.

Andrej Karpathy's full-stack ChatGPT-style training and inference recipe you can run end-to-end for $100 on a single 8xH100 node.

Andrej Karpathy's experiment: an ensemble of LLMs debating your hardest questions and arriving at a synthesized answer.

Andrej Karpathy's LLM training implementation in pure C/CUDA — small, readable, performant reference for understanding GPT training from the metal up.

Andrej Karpathy's Llama 2 inference in one file of pure C — minimalist reference for understanding LLM inference end-to-end.

Andrej Karpathy's minimal PyTorch re-implementation of GPT — ~300 lines of clean training code for understanding transformer language models.

Andrej Karpathy's minimal, clean implementation of Byte Pair Encoding (BPE) — the tokenization algorithm used by GPT and most modern LLMs.

Andrej Karpathy's tiny scalar-valued autograd engine and neural net library on top — PyTorch-style API in ~100 lines for teaching backprop.

Andrej Karpathy's quick illustration of reading books together with LLMs — annotate, ask questions, discuss as you go.

Andrej Karpathy's tool that renders any git repo into a single static HTML page — perfect for human reading or feeding the whole codebase to an LLM.

Andrej Karpathy's project analyzing decade-old Hacker News discussions in hindsight using LLMs to score predictions and ideas.

Andrej Karpathy's autoregressive character-level language model for generating more things — names, words, anything — from a small training set.

Fast BPE tokenizer training in Rust by Andrej Karpathy. The missing tiktoken training code.

Super simple calorie counter web app — a vibe-coded daily tool from Andrej Karpathy.