like Sang Heon Lee. And so, what I want to talk about is how we teach reinforcement learning, and do it the same way that we've been teaching hacking for a while. And it really breaks down into two different axes. The first thing when designing these sorts of tasks for people is to look at target difficulty. There's a spectrum of different challenges that you can look at from toy problems through CTF and synthetic problems, all the way up to hardened targets. The second axis for teaching machines to hack is really looking at exploitation difficulty. For example, when we look at a toy program, we may start looking at the sort of skills it needs to acquire to be able to hack that. For example, if you have a toy program and it has a bug, can the LLM figure out where the bug is? Can it then prove that it knows where it is by triggering a crash or some other fault in the program? But of course, hacking is not just crashing a program. We want to take control of that program. That's the beautiful thing about hacking. It's bending computers to our will. It's what makes it unique in the sciences. So you look at things like, "Hey, there's a flaw in that program. Can I use that to do arbitrary read writes in memory? Or even to do a full arbitrary code execution exploit?" And so if you remember nothing else from this talk, it's really the way that we teach LLMs, whether it be frontier models like Anthropic or private models that you're tuning in your house, you follow these two axes where you're trying to come up with a set of tasks that increase in target difficulty along one, and then you're teaching specific cybersecurity skills on the second. In other words, hacking is really a ladder. And this is what actually matches cybersecurity so well to reinforcement learning. We have a ladder of tasks and we typically end up with a good oracle for whether they can achieve that task. And so you can start to measure whether your model is learning the right set of capabilities. So this talk is really divided into three parts. The first one is to talk about vulnerability discovery. And when we talk about vulnerability discovery, what we're talking about is in the variety of different programs that you encounter in real life, how do you design oracles that are correct for determining whether or not a model has successfully been able to detect that vulnerability? And what's interesting is several of the cybersecurity benchmarks out there were amazing first-generation pieces of work, but they have a critical flaw where the model will actually stop learning after it finds the easiest vulnerability. And that can prevent them from getting smarter.