- Category
- AI Tools
- Pricing
- Open Source
- Type
- TOOL
- Builder
- karpathy
- GitHub
- 10.6k stars
- Added
- May 22, 2026
About
Andrej Karpathy's minimal, clean implementation of Byte Pair Encoding (BPE) — the tokenization algorithm used by GPT and most modern LLMs.
Why it made the leaderboard
Andrej Karpathy's minimal, clean implementation of Byte Pair Encoding — read it to actually understand the tokenization algorithm behind GPT and most modern LLMs instead of treating it as a black box.
Tags
bpetokenizerkarpathyllmeducation
Tech Stack
Python
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.