Anthropic's Constitutional AI uses written principles to guide model-generated training feedback. In the 2022 research, a model first critiques and revises responses, and those revisions become training examples. A second phase uses AI judgments between pairs of responses to train a preference model, whose scores guide reinforcement learning. This reduces reliance on people individually labeling harmful outputs while retaining human-selected principles as the basis for those judgments. The authors report an assistant that could explain objections to harmful requests while remaining engaged with the conversation.