Our Framework For Developing Safe And Trustworthy Agents

Anthropic's August 2025 framework for developing agents says human control must continue after a user supplies a goal. An agent is software that pursues a task on its own once given an objective, and Anthropic's expense-management example separates spotting overspending on software subscriptions from cancelling them: before it cancels or downgrades anything, the company would likely want a human to approve. Visibility into an agent's plan lets users correct its assumptions or data, while approval gates govern consequential steps. Anthropic proposes combining this oversight with autonomy for routine work, with the right balance varying by situation. The post describes principles and Anthropic's own practice, and it states that reliably measuring whether agents act as humans intend is still unsolved.