But here's the first problem. We have a scarce context window. If you take Fermat's Last Theorem as example, what it take for the mathematician was over 10 years time of reading paper, writing down thoughts in the scratch pad, or taking a walk to generate the creative ideas. If we convert to token, that's probably tens of a billions of even hundreds billions. But where we are now, just a 1 million token context window. So one solution is use compaction. So what it essentially does is generate the token until the end of the context window, summarize, and then on top of that generate more tokens. And the beauty of applying RL in the situation is kind of like kill two birds with one stone. You apply RL to the compaction and also the task. But here's the problem. With long horizon, there are three issues. The first is the gradient variance scales with the length. And the second is a sparse reward. And you have this credit assignment problem. And finally, there's also variable length of the trajectory that adds to the problem of optimization. So to solve this issue, we can apply critics, which is the value model. And value model can reduce the variance and also have a couple advantages, such as on the trajectory level that fits compaction very well, and also encourage the batch diversity. And also, I'll talk later on bootstrapping. Basically, get signal before the end of the episode. But, the downside for this is that um it's more complicated than GRPO, and basically, you have to train another value model alongside with the policy model. And there's some tools to help with the context limitations, such as a file system tools, which essentially like a scratchpad for AI to write to the uh reasoning thought. And self-search tools, which allows agent to search over the previous trajectory. And then you have archive tools. In the case like auto research, you can build upon your previous result. But, we have to be careful. In other scenario, you don't want an AI to cheat by just to grab the previous answer without thinking.