RLHF Book & Post-training Course
- Source
- interconnects.ai
- Author
- Nathan Lambert
- Date
Context
Nathan Lambert's book on post-training, the work that shapes a pretrained language model into a useful assistant, now has a free lecture series alongside it. The announcement introduced lessons progressing from training foundations to the mathematics and implementation of reinforcement learning. The useful connection is between understanding why a training method works and seeing how to implement it.
The course page provides recordings and accompanying materials organized around book chapters. It describes an audience roughly at early graduate level and includes a foundations refresher, giving learners a way to revisit prerequisites alongside the more technical material.
- The Relative Adoption Metric normalizes downloads by model size and by time, scaled so a score above 1 means a model is on pace to become one of the ten most-downloaded models ever in its size class. One number, messy landscape.
- The ATOM report tracks GPT-OSS's rise, market share, and the pull of China's mid-tier labs — Moonshot, Z.ai, MiniMax — arguing much of the open-model contest is being decided below the frontier tier.
- TurnWise measures the gap between single-turn and multi-turn ability: models that look strong on one-shot benchmarks degrade across a conversation, and the paper covers how to build training data that closes it.
- Most RL on language models is on-policy but naive — the model learns from recent attempts in its weights, not in its . The meta-RL search paper feeds prior rollouts back in as context so later attempts inherit what earlier ones found.
- RLHF — Reinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.
- alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
“The RAM score is designed so that a score >1 indicates a model is, at that point in time, on track to be a top 10 most downloaded model of its size category, ever.”
“The goal of this book was to write the book I wished I had when I was getting started in post-training language models.”
“Research is the only early warning of some big changes coming.”
Checking sign-in…
Loading comments…