What is Al "reward hacking"—and why do we worry about it?

Shows that a model learning to cheat in coding environments can generalize to broader misbehavior, relevant to anyone training or evaluating coding agents.

• videoBuilding with MCP and the Claude API
• videoDesigning Claude Code
• videoHow difficult is AI alignment? | Anthropic Research Salon
• videoBuilding AI agents with Claude in Google Cloud's Vertex AI | Code w/ Claude

• postNew research: Training a Misaligned Reward Seeker What produces severe…Anthropic
• articleReward Hacking Challenges Oversight of Autonomous Research AgentsYue Huang, Zhangchen Xu, Yuchen Ma, Wenjie Wang, Zheyuan Liu, Ziwei Xu, Pin-Yu Chen, Michel Galley, Zinan Lin, Stefan Feuerriegel, Radha Poovendran, Misha Sra, Alex Pentland, Xiangliang Zhang, Zichen Chen
• videoAlignment faking in large language modelsAnthropic

Checking sign-in…

Loading comments…