offline when you're kind of creating these rubrics. You can do things like red teaming with adversarial prompt optimization to to kind of explore for backdoors. Then you can look for traces and spend compute mining these traces for understanding was this the reward a reward hacker? Was this actually kind of in the spirit of the task? And I think these things can kind of feel like reward hacking can kind of sneak up on you if you're not careful for it, but in many cases the basic simple things actually work quite well where if the reward hacks are the sorts of things where a human can look at them and be like, oh yeah, that's a reward hack. Judges are often not quite good at doing this as well. They just don't necessarily tell if you tell the model not to do this, it won't necessarily do it in the rollout, but in hindsight you can reflect on this and spend compute to kind of especially if you are collecting these over time and you are building up your corpus of examples of reward hacks, you can understand the sorts of things that go wrong and address this by kind of again spending inference compute on refining your implementation, refining your rewards. As well as validating these by training. And so we find that in many cases you can do a lot up front, but also also there are things that don't show up until you actually like start doing RL. And so part of this is folding in training experiments themselves into the process of environment design where you can you can do small runs with individual models on like one environment and and see what happens and you can understand that the behavior changes. You can have metrics that log the the types of tool calls that are being done that are like judges asking questions about the the traces to understand how behavioral patterns are changing. And all of these are very useful ways of kind of getting something from nothing and using compute as the thing that allows you to refine your understanding. And ultimately what you want is to surface the most important pieces up to the human, the the highest level of questions about what is actually going on, what is the goal so that the all this is deferring to the human for the most important pieces of like actually employing expert to say this is good, this is bad, this is what I want, this is not what I want. And uh these are all the ways that we kind of gain confidence in the environments.