Traces find failures; offline simulation makes them repeatable

benchmarks that are close to production and uh specific domains. So, a lot of the time when people say about agent evaluation, they're focused on traces. And traces are very useful. The usual traces like you can see an example on the screen. It basically shows, "Okay, here is the input prompt. Here are the actions that agent took, and here is the agent output." And then relation can analyze it and and say, "Okay, was agent successful or not? Was there any edge case?" Uh so, it is useful to find failures in production, but it's hard to test different variants. You can run AB testing, and that's one way of checking different agent configurations, but it's hard to make sure that everything is repeatable because you will get different database state, different tool versions, and so on. So, never fully compare apples to apples. Offline simulation turns traces into repeatable experiments. Now you take production traces, you construct tasks, and then you can run simulation benchmark um with different agent configuration offline. And you can compare agents using different metrics, not just success rate, but cost, latency, and retries. And you can run those in parallel.