Clip transcript
environment. The main challenge with the environment is that effectively it has to be mini production, but you don't want to run full production for every experiment. So, you want to make sure that uh your database, API service, tools, and files match production, just as like previous speaker mentioned like that you don't want your agent to know that it's running within simulation. So, it has to be real. One thing, though, you cannot put real user in your simulation task. So, you can simulate the user. In this case, that becomes effectively LLM with its own prompt, which with additional context that can mimic human behavior and interaction with your system. So, in the port system that all these things exist in the environment, and verifiers just interact with the environment afterwards. There are certain patterns to make sure that you can organize environment this way. And think about it as like how you construct integration tests. Basically, how you give effectively diff environment to your agent. You don't run the full production database. You have a certain snapshot. You can run side containers sidecars in this case. So, your uh agent runs in one main uh environment, but there are other containers which contain API services, databases, MCP tools, and so on, available to your agent. You don't need to have like full production API services, you can mock them. I already mentioned simulated users. Uh and uh one important piece here is multi-step to handle long horizon task, if your agent needs to handle task that span hours. You want to ensure that you have intermediate like steps, and for each step you have separate prompt, separate verifiers, and uh you can uh finish simulation early if you see agent failing. And basically that enables you to simulate long-running horizon tasks.