Vibeleaderboard
← All Intel
Intel / article

AI Safety: Not Optional, Not Later

Source
arxiv.org
Author
Qinghua Lu, Yoshua Bengio
Date

Context

An 's proposed action should pass checks outside the model before it changes anything consequential. That is the central mechanism in AI Safety: Not Optional, Not Later. The proposed architecture separates planning from the software that controls tools, permissions and network access. Supervisory models and independent checks assess a proposed action; the controlling software can then permit it, require revision or refer it to a person.

The design also places monitoring outside the agent's control loop and preserves records of actions and interventions. This gives operators ways to constrain behavior and investigate failures even when the agent's own judgment is unreliable. These are proposed safeguards, rather than measured evidence that the architecture prevents incidents.

Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Recommended reads
Comments

Checking sign-in…

Loading comments…