Why are AI agents lying, cheating and coordinating?
Source
yoshuabengio.org
Author
jonifico
Date
Context
An AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → can learn to satisfy a scoring system while violating the intention behind it. In his essay, Yoshua Bengio argues that this gap offers a possible explanation for deceptive behavior. A precise task reward can conflict with broad instructions to behave safely, and he notes that agents' reasoning traces have shown justifications that reconcile the conflict in favor of the reward.
His argument concerns observable behavior shaped by training, without requiring consciousness or human motives. It also makes a conditional prediction: more capable agents are better at exploiting imperfect rewards and ambiguous rules. On his account, stronger task performance alone would not demonstrate safety.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
