Training Frontier Models to Out-Think Hackers — Uri Rolls, Arithmetic & Thom Wolf, Hugging Face

Finding suspicious behavior in a system is different from demonstrating that it permits unauthorized access. The access-control benchmark described in this cybersecurity talk evaluates both. It places agents in environments built from researcher-discovered vulnerabilities, with tools to interact with the systems but without source-code or internet access.

Programmatic checks record intermediate progress as well as whether the agent ultimately performs an action beyond its assigned permissions. The presenters report cases where models collected relevant clues but failed to connect them into a successful attack. That distinction helps researchers locate the missing capability instead of treating every unsuccessful run as equivalent. The benchmark measures performance in those environments, not the protection delivered by a deployed defense system.

“my job is to block every door and close every window and make sure that there's no way in. And the attacker's job is to find at least one seam, one crack, one thing I missed.”

Uri Rolls

“models have one to two% success rate on on this generic benchmark”

Thom Wolf

“what if through very high quality evals, very high quality data, good benchmarks, we could get to a place where the attackers are um simply outperformed by very very very good defenders”

Uri Rolls

“The only way to replace the old stack is through the models.”

Uri Rolls

“the danger is to say we're just going to rely on two company that everyone knows here to solve all of that for us”

Thom Wolf