AI models can provably hide arbitrary capabilities
Source
arxiv.org
Author
sbulaev
Date
Why it matters
Shows that evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →-based safety audits cannot rule out hidden capabilities in a backdoored model, even with full weight access. Teams relying on evals for supply-chain trust should weigh that limit.
Terms in this piece · Glossary
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
articleReward Hacking Challenges Oversight of Autonomous Research AgentsYue Huang, Zhangchen Xu, Yuchen Ma, Wenjie Wang, Zheyuan Liu, Ziwei Xu, Pin-Yu Chen, Michel Galley, Zinan Lin, Stefan Feuerriegel, Radha Poovendran, Misha Sra, Alex Pentland, Xiangliang Zhang, Zichen Chen
article[AINews] How to steal a Reasoning Tracewww.latent.space
articleOpenAI’s accidental cyberattack against Hugging Face is science fiction that happenedSimon Willison