is kind of an undefined problem. And then especially with RL, we have to be very careful about reward hacking. And so reward hacking is when you have a kind of loose proxy for your objective that is undefined at the boundaries. And then models, if you train with RL, they can learn to exploit this and find weaknesses where there's some path towards a climbing the the reward that doesn't actually give you what you want. And so really the goal of what we would hope all of this builds into is continual learning, which is a big buzzword that I think a lot of people like talking about in many different ways, but I'm going to use it to mean a very particular thing, which is that we want models to be deployed in relatively realistic complex, messy settings and to be able to learn as they go where they are doing things, they are making mistakes, they are then able to observe and catch these mistakes after they happen, and use this to not do the same thing again. In some cases people want to try to do this at the harness layer of the front layer. But ultimately you want a system that can evolve autonomously to be able to get better over time with humans in the loop at that right level of abstraction. And I think currently the level of abstraction for doing this is far too low for it to be practical for most people. And so this means we need new methods to be able to uh automatize as much of the difficult processes as possible. And many of these actually are automatizable. They just are kind of difficult problems to solve. And so there's a few techniques you can you can use to kind of start making progress here, but one of the goals here is to do online reinforcement learning so that you can kind of iterate on this process as you go, as well as beyond just RL.