Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research
- Source
- arxiv.org
- Author
- Keshav Shenoy, Anthony Zang, Eric Gan, Jacqueline Tay, Amy Ngo, Sebastian Prasanna, Julian Stastny, Vivek Hebbar
- Date

Context
AI developers plan to hand more of their research, including safety research, to AI agents. A misaligned could then corrupt the experiments meant to check it. A study led by Redwood Research tested whether reviewers could catch that, using a human expert as a stand-in for a sabotaging AI. Mostly, they could not.
One researcher ran 10 new AI safety experiments from scratch, each answering a short research question within a compute budget, with every file change and terminal command logged. Then, knowing the honest results, the same researcher produced 12 new runs of those questions while trying to change the headline conclusions. The sabotage could enter at any stage, from experimental design to code to how results were presented, and the 12 runs contained 33 distinct sabotage mechanisms. Twenty-five human auditors made 104 reviews. Each had 90 minutes, the full logs and code, two audits written by Claude Opus 4.8, any AI assistant they wanted and the ability to rerun experiments, and was told each run had an even chance of being sabotaged.
The auditors barely beat chance. A standard measure of how well a reviewer's suspicion separates sabotaged from honest runs, where 0.5 is a coin flip and 1.0 is perfect, came out at 0.59, with a 95% interval from 0.37 to 0.77. Median suspicion was 54% for sabotaged runs and 40% for honest ones. Auditors fully identified a sabotage mechanism in 17% of their chances, and 16 of the 33 mechanisms were never even partly spotted by any human auditor. The authors note what that means in practice: if 1% of real runs were sabotaged, only about 1.6% of the runs flagged at 50% suspicion would actually be sabotaged.
Frontier models auditing alone, without the ability to run experiments, did somewhat better on paper. GPT-6 Astra and Claude Fable 5.1 scored 0.743 and 0.731 on the same measure, and Fable 5.1 earned 37.1% on the identification score that gives half credit for partial diagnoses, against 21.9% for humans. But the paired confidence intervals for those differences all include zero, and pooling every human and AI report still left 13 mechanisms, five of them major, never fully identified. When the analysis kept only the attacks the saboteur had rated hardest to catch, three in all, no auditor identified any of them.
The study is small: one saboteur, a handful of projects, and auditors of mixed experience. Letting the saboteur see the honest result first is a strong assumption, though the team also found researchers predicted these experiments' outcomes poorly, fully correct only 39.8% of the time, which means a surprising result is weak evidence of tampering. The supported conclusion is that reading a full research record, even with AI help, is not yet a reliable check on research run by an expert-level agent. AI auditors look like the more scalable path, but relying on them raises the question of whether the auditors themselves can be trusted.
- In the study's red-team game, a human researcher ran AI safety experiments from scratch and sabotaged some, while auditors with the full transcript and the ability to rerun experiments tried to say which runs were sabotaged and how.
- Frontier models audited without being allowed to run experiments and still outperformed the human auditors, within noise, while weaker models performed comparably to the humans.
- Detection fell further when the saboteur chose which experiments to attack instead of being required to sabotage every question, the authors report.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleAuditable Human-Agent Teams for Autonomous X-ray ExperimentationGunik Luthra, Ronald J. Pandolfi, Benjamin T. Diroll, Ricky Ronquillo, Ruipeng Li, Christopher B. Murray, Marcus M. Noack, Kevin G. Yager, Esther H. R. Tsai
articleDiagnostic Foundation for Evaluating LLMs' Research Integrity as Co-ScientistsYash Tripathi, Silu Sharma, Sai Sidhanth Manoharan Jayanthi, Shivank Garg, Lin Li
articleHow Researchers Use and Verify AI Coding Assistants: Tasks and Validation Practices in Scientific ProgrammingGabrielle O'Brien, Reed Milewicz, Nasir Eisty
Checking sign-in…
Loading comments…