Scaling up Continual Learning — Ronak Malde, Trajectory
- Source
- AI Engineer
- Author
- AI Engineer
- Date
- distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Long-horizon training has a specific, reproducible failure mode here, along with the corrections that keep it from collapsing into hedging.
“Scale on policy self distillation to trajectories with a hundred tool calls and the model collapses into hedging. The tokens it learns to favor fill up with wait, but, and maybe, until, as Ronak Malde puts it, everything just turns into maybe.”
AI Engineer
“At the frontier there is no smarter model to distill from, so you make the model its own teacher: put privileged information, a hint, in the teacher's prompt, and match the log probs of the student that never saw it.”
AI Engineer
“GRPO gets on policy sampling but explodes parallelism and collapses feedback into one sequence level score, which he compares to being handed 87 out of 100 on an essay and told to work out why.”
AI Engineer
“Self distillation gets all four, and it optimizes across the entire vocabulary at every token instead of sharpening the one that was sampled, which is why it keeps climbing past where GRPO plateaus while tokens to solve go down rather than up.”
AI Engineer
“The failure modes are the useful part: step level KL weighting to handle divergence, and residual guidance for hint leakage, the self distillation analogue of reward hacking, where a hint containing the answer teaches the model to state it and back fill the reasoning afterward.”
AI Engineer
videoDistill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash
videoTeaching LLMs to Speak Spotify — Yves Raimond & Jacqueline Wood, Spotify
videoWhy LLM Recommenders Will Be AI's Biggest Consumer App — Devansh Tandon, Meta
videoWorld Models Need Causality, Not Pretty Pixels — Christopher Manning, Moonlake AI
Checking sign-in…
Loading comments…