- Category
- Education
- Rank
- No. 243Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- karpathy
- GitHub
- 17.4k stars
- Date
About
Andrej Karpathy's tiny scalar-valued autograd engine and neural net library on top — PyTorch-style API in ~100 lines for teaching backprop.
What it does
It records ordinary arithmetic as a graph, then walks that graph backward to calculate how each input affected the result. Compact neuron, layer, and multilayer perceptron abstractions combine those scalar operations into trainable classifiers.
Why it's ranked here
Pick it for understanding automatic differentiation through readable, working code. The implementation exposes graph construction and gradient propagation directly, while tests compare forward results and gradients against PyTorch. The included classifier and graph visualization notebooks make the core mechanics observable.
What's good
Arithmetic expressions build the computation graph naturally, including addition, multiplication, numeric powers, division, negation, and ReLU. Backward propagation follows a topological ordering and accumulates gradients. The neural network layer stays small enough to connect weights, activations, parameters, gradient clearing, and multilayer composition without hiding the machinery.
Tradeoffs
Every graph node holds one scalar, so neurons expand into many individual additions and multiplications. The operation set is deliberately narrow, and the neural library only supplies basic feedforward components. Visualization needs Graphviz, while tests need PyTorch. The repository itself points advanced transformer work toward a different, more efficient engine.
How to use it well
Use it as executable study material for backpropagation, or as a compact reference when teaching computation graphs and basic neural network training. Work through arithmetic first, inspect gradients, then follow the classifier and visualization notebooks. Choose another system for tensor computation, broader model architectures, or efficiency-focused training.
Technical notes+
micrograd/engine.py defines Value, stores data and grad, captures parents in _prev, attaches per-operation _backward closures, builds a topological order, and traverses it in reverse during backward(). micrograd/nn.py builds Module, Neuron, Layer, and MLP over Value objects. test/test_engine.py checks forward values and gradients against PyTorch. demo.ipynb trains a two-hidden-layer binary classifier with max-margin loss and SGD. trace_graph.ipynb traverses internal graph state and renders data, gradients, and operations through Graphviz. setup.py packages the Python modules with setuptools and requires Python 3.6 or newer.
Observed
- License
- MIT License
- Primary language
- Python 3
- Installation
- Published package installable with pip install micrograd
- Packaging
- Setuptools package requiring Python 3.6 or newer
- Interface
- Python library for scalar automatic differentiation and feedforward neural networks
- Platform support
- Declared operating-system independent
- Tests
- Unit tests compare forward values and calculated gradients with PyTorch
- Examples
- Repository includes notebooks for classifier training and Graphviz computation-graph visualization
Read from README.md, setup.py, micrograd/nn.py, micrograd/engine.py, test/test_engine.py, LICENSE, demo.ipynb, trace_graph.ipynb.
What it can do
Perform automatic differentiation on scalar values
Mathematical expressions with scalar variables → Gradients of the expression with respect to input variables
Build neural network models
Network architecture specifications (layers, neurons) → Neural network model object
Train neural networks using backpropagation
Training data and neural network model → Trained model with updated weights
Compute forward pass through neural networks
Input data and neural network model → Predicted output values
Calculate loss functions for training
Predicted outputs and target values → Scalar loss value with gradient information
Demonstrate backpropagation algorithm step-by-step
Simple neural network example → Educational trace of gradient calculations
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.