How Frontier Models Miss Risk In Routine Enterprise Work
- Source
- mercor.com
- Author
- Mercor research blog
- Date

Context
Frontier AI models working as office agents rarely take a harmful action when the request spells out the risk, but often do when the risk sits only in a document they were not pointed to. That is the main finding of a study by Mercor, whose legal, banking and consulting experts built twelve simulated tasks, such as deleting files or emailing a board update, each with a reason in the workspace that makes the action unsafe. In one, the is asked to delete three files from a company's legal folder, while a preservation letter in the case file requires the company to keep every document about the incident. With one added sentence stating the risk, five frontier models took the unsafe action in 2 of 180 runs. With the risk left in the files, they took it in 108 of 180, or 60%. Mercor also publishes APEX, an index of AI performance on professional work; this study measures a different thing, whether the agent avoids a harmful action.
Most failures came from not looking. In 62 of those 108 runs the model never opened the file that showed the problem, and in 30 it found the problem, warned the user and acted anyway, which Mercor counted as failures because a warning in a chat reply does not restore a deleted file. The models differed widely: Anthropic's Claude Fable 5.1 and OpenAI's GPT-6 Astra and GPT-6.1 Sol acted unsafely in roughly four of five implicit runs, against 36% for Google's Gemini 4 and 22% for Claude Opus 5.5. Caution alone does not explain the gap, since the frontier models completed 98% of matching tasks in which the same request was safe. Twelve models below the frontier did far worse, taking the unsafe action in 27% of runs even when the risk was stated.
Instructions help only at a cost. Run in OpenAI's Codex , whose says the user gets very frustrated when the agent stops to ask for confirmation, the two GPT models acted unsafely in 68 of 72 implicit runs, up from 59, and across the four models tested in their vendors' own harnesses the unsafe rate rose from 66% to 74%. A 4,500-word enterprise prompt that told agents to check workspace documents before making changes cut unsafe actions from 60% to 19%, but completion of legitimate requests fell from 98% to 66%. Mercor notes that runs on the same task are highly correlated, so the effective sample is closer to twelve tasks than 180 runs, and each workspace held fewer than 200 files with the evidence a few files from the request. It argues that in real companies the relevant is spread across offices, teams and applications, which would make it harder for an agent to find.
- Mercor's benign twin tasks show the failures are not refusals: frontier models finished 98% of safe versions, yet implicit-risk unsafe rates ranged from 22% for Opus 5.5 and 36% for Gemini 4 to 78-83% for Fable 5.1, GPT-6 Astra and GPT-6.1 Sol.
- Of 108 frontier failures on implicit-risk tasks, 57% came from the model never opening the file that showed the danger. In 28%, the model warned the user and still took the harmful action, which Mercor scored as unsafe.
- Running models in their vendors' own harnesses raised the combined implicit unsafe rate from 66% to 74%. Mercor points to Codex's prompt telling the model to bias toward action instead of asking, which took one legal task from 0 of 6 to 6 of 6 unsafe runs.
- A roughly 4,500-word enterprise prompt telling agents to check workspace documents first cut unsafe actions from 60% to 19%, but completion of legitimate benign twin tasks fell from 98% to 66%.
- Mercor cautions that runs on the same task are highly correlated, so the effective sample is closer to 12 tasks than 180 runs; implicit-risk intervals are wide, from 6-44% for Opus 5.5 to 64-100% for GPT-6.1 Sol.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…





