Clip / AI Tools

Evaluating long runs: intermediate submissions and reward-hack checks

From MiniMax's Open Weight Model Strategy and Inference Stack · ≈15:28

Shows a practical pattern for grading multi-hour agent tasks — score intermediate submissions, hold out validation/test to catch hacking, and build internal evals from your own workflows.

What’s in it

Shows a practical pattern for grading multi-hour agent tasks — score intermediate submissions, hold out validation/test to catch hacking, and build internal evals from your own workflows.

Watch this moment

Clip transcript

are there like a intermediate things that you can also look at? >> Yes, we do. For these tasks, they are there are iterations, right? So, the model can submit several times, and we would evaluate each of them. Some of them some of the times the models would hack, and we do do like validation and test with for it to test if it's really improving on the performance or it's hacking. And also, we design our internal evaluations. So, for example, for the release of 2.7, we touched a bit on self-evolution, right? So, we're actively using the model to improve the speed of development internally, which like out of it out of it we can build our own evaluations that are closely related to our own work that we can evaluate the models on.