← All IntelClip / AI ToolsReference stack for building RL environments and post-training agents
From Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs · ≈17:13
“So that also works pretty well for updating the system prompts and also the harnesses.”
What’s in it
- Maps out a 3-layer stack for building RL environments and post-training agents
- Explains why long-horizon rollouts need checkpointing and snapshot rollback
- Introduces GEPA, an LLM-driven method for auto-optimizing prompts via reflection
Clip transcript
of converging on something very similar so if you think about uh the the stack that is needed to say not just curate these RL environments but to post train models one of the things you need is obviously handle on like how do you build these RL environments how do you measure the quality how do you track the different versions and so on so that's one of the layers and below that you want various infrastructure to uh um sand use sandboxes, right? to spin up the rollouts to to spin up the sandboxes to generate rollouts and especially if you have long horizon rollouts then maybe at some points you need to do a checkpointing and then you need to be able to snapshot or roll back to something else right so that's the other uh the lower level u you know compute and orchestration and at the top I have been giving examples on post training so there is all this uh layer around like how do you do SFT how do you do RL and so on but there is also this method called Japa which is around uh which is on prompt optimization. I don't know if you if you guys have heard of it but you can use LLMs itself to uh to to kind of optimize the prompts based on reflection. Um so that also works pretty well for updating the system prompts and also the harnesses. So this is kind of I feel like you know the the new architecture or the new reference uh stack for how how at least we are building and how many others are building um the the stack on how to build the RLMs and then also post train agents. I think with that uh I will uh
Comments
Sign in to comment.
Loading comments…