Training language models to follow instructions with human feedback
Source
arxiv.org
Author
Long Ouyang et al.
Date
Why it matters
The gap between a raw LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → and an assistant is this paper: alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition → to human preference beat raw scale for usefulness, with a 1.3B InstructGPT preferred to 175B GPT-3. It is the direct ancestor of every chat model's post-training.
Terms in this piece · Glossary
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
RLHF — Reinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
