AI Safety: Not Optional, Not Later
- Source
- arxiv.org
- Author
- Qinghua Lu, Yoshua Bengio
- Date

Context
An 's proposed action should pass checks outside the model before it changes anything consequential. That is the central mechanism in AI Safety: Not Optional, Not Later. The proposed architecture separates planning from the software that controls tools, permissions and network access. Supervisory models and independent checks assess a proposed action; the controlling software can then permit it, require revision or refer it to a person.
The design also places monitoring outside the agent's control loop and preserves records of actions and interventions. This gives operators ways to constrain behavior and investigate failures even when the agent's own judgment is unreliable. These are proposed safeguards, rather than measured evidence that the architecture prevents incidents.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleAI Evaluation Should Work With HumansJan Kulveit, Gavin Leech, Tom\'a\v{s} Gaven\v{c}iak, Raymond Douglas
articleAI Policies: Help or Hindrance? A Software Developer's PerspectiveSamuel Ferino, Rashina Hoda, John Grundy, Christoph Treude, Hashini Gunatilake
articleMLLMs Fail to Refuse when Using Tools AgenticallyRikiya Takehi, Ryo Hachiuma, Shaona Ghosh, Dan Zhao, Yu-Chiang Frank Wang, Yusuke Hirota
Checking sign-in…
Loading comments…