Training Agents: Live tutorial on how to fine-tune a coding agent for continual learning

This is the first live session of a Hugging Face series on training agents, and the most useful thing it explains is what supervised fine-tuning, or SFT, on agent traces can and cannot do. A trace is the full record of an agent session: the user's request, the model's replies, the tool calls it made and the results that came back. The presenters fine-tune a small open Gemma model, about 2 billion parameters, on a public set of traces from real coding sessions that, according to the presenter, were produced with Claude Opus 4.5 inside an agent harness.

The core mechanism is masking. SFT is still next-token prediction, as in pre-training, but each example is a prompt and a completion. The presenter walks through a short PyTorch version in which every token the user wrote is labeled -100, which tells the loss calculation to ignore it, so the model's weights only learn from the assistant's side of the conversation. That is how a small instruction-following model picks up the stronger model's habits, including how it formats tool calls across many turns. TRL, Hugging Face's training library, does the same thing with an option that computes loss only on assistant responses.

The presenters are blunt about the limit this creates. The run picks its best settings by loss on held-out traces, which they describe as rewarding "the best imitator," not the best agent. The presenter adds that the small model cannot learn to do better than the traces it copies and has its own capacity ceiling. The realistic goal, in his framing, is narrowing: train on good traces for one task, such as triaging pull requests, accept that other skills may dip, and use a broader benchmark to check you have not broken the model's general coding ability. He also says the evaluation process the agent built was not necessarily the best, and that the two coding benchmarks in the prompt are probably a bit saturated, though still relevant here.

The stream is also a demonstration of handing the whole job to a coding agent. Instead of writing the training script, the presenter gives the agent a prompt listing the model, the dataset, a hyperparameter sweep on Hugging Face's cloud jobs, live metric tracking and evaluations, plus skill files that tell it how to use those tools. The agent runs cheap smoke tests before launching paid jobs and recovers from errors along the way; this is a single live demo, not a measured result. Reward-based training, which the presenters say is needed to go beyond imitation, is left for later sessions such as the series' reinforcement learning session.