Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute
Source
AI Engineer
Author
AI Engineer
Date
Terms in this piece · Glossary
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Why it matters
It shows the actual moving parts of keeping a deployed model learning from its own production traffic.
Key quotes
“if the agent learns the exact environment distribution, why don't we just use that for our training?”
Raymond Feng
“it makes a lot of sense that because there's so many potholes, the model doesn't want to run for that long uh because it might fall in a pothole and then get a zero reward for the rollout”
Raymond Feng
“it will actually just be incentivized to like abuse the tool calls and just like call a lot of them in quick succession and try to time out the sandbox”
Raymond Feng
“you're kind of playing a game of Whac-A-Mole where as soon as a new thing pops up, you need to scramble and like create new data or new environments”
Raymond Feng
“AI is at the cusp of a new period in which experience will become the dominant medium of improvement and ultimately dwarf the scale of human data used in today's systems”
