Reward Hacking in Reinforcement Learning
lilianweng.github.io- Category
- Other
- Type
- ARTICLE
- Builder
- @lilianweng
- Added
- Jul 21, 2026
About
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task. Reward hacking exists because RL environments are often imperfect, and it is fundamentally challenging to accurately specify a reward function. With the rise of language models generalizing to a broad spectrum of tasks and RLHF becomes a de facto method for alignment training, reward hacking in RL
Why it made the leaderboard
If you're doing RLHF or RL fine-tuning of language models, this explains how agents exploit reward-function flaws — modifying unit tests to pass coding tasks, sycophantically mirroring user preferences — so you can anticipate and mitigate these failure modes before deployment.
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.