Explains the training recipe behind principle-guided model behavior: a written list of rules drives self-critique and AI-preference labels, replacing most human harm labeling.
Key takeaways · AI-distilled
Constitutional AI trains a harmless assistant without human labels for harmful outputs: the only human oversight is a written list of principles, the constitution.
In the supervised phase the model critiques and revises its own responses and is fine-tuned on the revisions; in the RL phase a model compares pairs of samples, those AI preferences train a preference model, and that serves as the RL reward (RLAIF).
Anthropic reports the result is harmless but non-evasive, engaging with harmful queries by explaining its objections, and that chain-of-thoughtHaving a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.Full definition → reasoning in both phases improved human-judged performance and transparency.
Terms in this piece · Glossary
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
