SLMs as Multi-Agent Routers: A Progressive SFT and Reinforcement Learning Approach
Source
Gayathri V Kondapalli, Alexander Ng, Hirsh Pithadia, Rahul Monish, Harvey Yorke, Amir Kayhani
Author
Gayathri V Kondapalli, Alexander Ng, Hirsh Pithadia, Rahul Monish, Harvey Yorke, Amir Kayhani
Date
Key takeaways · AI-distilled
Intent-based AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → routing has a blind spot: it cannot detect when a topically aligned retrieval agent returns low-relevance results, because it never sees the retrieved content.
The fix: train a small LM with SFT then RL to jointly select the agent AND generate structured tool-call parameters, rewarded hierarchically on retrieval relevance plus topic alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition →.
On deliberately mismatched agent-query cases the trained SLM hits NDCG@10 of 0.918 vs 0.539/0.490 for Nova Lite and Claude Haiku 4.5 routing on intent alone.
Overall it wins on both quality and cost: mean NDCG@10 0.771 (+0.18-0.22 over the LLM baselines) at 120ms mean selection latency - an 82% latency reduction.
Terms in this piece · Glossary
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
Why it matters
Shows that training a small router model with RL against actual retrieval relevance—rather than inferring intent—can outperform larger LLM-based routers on both accuracy and latency, a directly applicable pattern for multi-agentUsing several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.Full definition → systems that dispatch to specialized retrieval agents.
Key quotes
“Specialised retrieval agents typically surface higher quality results than general-purpose search, but selecting the optimal agent for a given query remains an open problem.”
“On a targeted subset of such agent-query mismatches, the trained model achieves an NDCG@10 of 0.918 compared to 0.539 and 0.490 for two LLM baselines (Amazon Nova Lite and Claude Haiku 4.5) that route on intent alone.”
