Crossing the Threshold: How OpenAI and Anthropic Approach Autonomy and Safety in Financial Agents
Source
latent.space
Author
Richard MacManus
Date
Context
A financial AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → can investigate a problem while leaving permission to change records with a person. In the reported conference example, Anthropic described an agent that investigates a trade discrepancy and proposes an order amendment for human approval.
The surrounding design identifies whose permissions the agent uses, defines its execution and approval boundaries, and checks its behavior after changes. Human review sits at the transition from analysis to a consequential action, giving the agent room to investigate while preserving control over the proposed amendment.
Key takeaways · AI-distilled
OpenAI's Joseph Kim described a 'threshold of usefulness' where a model is reliable enough to save time, and predicted that in 2027 finance will rely less on Excel and PowerPoint and agents will handle investment idea generation and trading.
Two Sigma's John Feminella warned that approval fatigue both slows teams and adds risk, and described composable controls including auto-mode classifiers that clear routine calls and escalate only consequential ones, with deterministic rules for clear cases.
Anthropic's Keon Shahab framed an agent trust layer in three parts: identity and permissions, a sandboxAn isolated environment where AI-generated code or agent actions run without being able to touch anything real.Full definition → approval layer before the agent acts, and checks that it still works as variables change. His trade-break agent proposes order amendments but does not make them.
Bloomberg's Andrey Rybka cited an agent that treated text in a marketplace review as an instruction to stock a product, and argued for deterministic controls outside the model: scoped credentials, tool-call checks, egress limits, audit trails, budgets and kill switches.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
sandbox — An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
