Vibeleaderboard
← All Intel
Intel / article

Constitutional AI: Harmlessness from AI Feedback

Source
arxiv.org
Author
Yuntao Bai et al.
Date

Context

Anthropic's Constitutional AI uses written principles to guide model-generated training feedback. In the 2022 research, a model first critiques and revises responses, and those revisions become training examples. A second phase uses AI judgments between pairs of responses to train a preference model, whose scores guide reinforcement learning. This reduces reliance on people individually labeling harmful outputs while retaining human-selected principles as the basis for those judgments. The authors report an assistant that could explain objections to harmful requests while remaining engaged with the conversation.

Recommended reads
Comments

Checking sign-in…

Loading comments…