careful about how to optimize your use between training and inference. And pipeline RL is a quite popular technique nowadays. So basically, it's a trade-off between off-policy and the GPU utilization. So traditionally, you let inference run towards the end and then you start to train the model. But in the case of long horizon, you have to wait until the inference finish. What the pipeline RL does is that you let the sequence to be generated and you start to train the model while there's a still more sequences being generated. And you see this created off-policy. But from the our experience, normally off-policy up to eight steps is okay. So, essentially, we made a trade-off between the off-policy and the GPU utilization. But, here comes the issue. As we the long horizon indicates, sometimes the inference would take weeks or even more. In that case, inevitably, it will goes beyond the constraint of the eight steps of our policy. So, your GPU have just to sit there idle and wait for it to finish. And if you don't want to wait, as I mentioned before, applying the value model allows you to bootstrap. What it means is that before the end of the episode, you generate expectation. It's like a dopamine in human brain. And that allows you to train the model. But, here's another trade-off. While you utilize the GPU fully, you introduce the value model bias. So, there's always a bit of trade-off in those solutions. And I want to also mention that in the long horizon, infrastructure is important, especially for the environment. And Open Review was a product is a platform by General Reasoning. If you're interested, you can check it out. openreview.ai. So, it's a place where host over 350 environments and with a single API endpoint. And we use this for our internal RL and also some frontier labs and new labs are using this.