← Glossary
Glossary · training
RLHF
Also: reinforcement learning from human feedback
Reinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.
A pretrained model will happily continue any text, helpful or not. RLHF adds a shaping step: humans compare pairs of model answers, a reward model learns those preferences, and the model is trained to produce answers the reward model scores highly.
This is a big part of why assistants follow instructions, decline harmful requests, and "sound" the way they do. Its known failure mode: models can learn to produce answers that look good to raters — confident, agreeable, polished — rather than answers that are right, one root of sycophancy.
In the index
- Reward Hacking in Reinforcement Learning
- Stanford CS329A Self-Improving AI Agents | Part 4 | Learning from Feedback with Tools/Code
- #490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
- DeepSeek, Reasoning Models, and the Future of LLMs
- Towards Understanding Sycophancy In Language Models