Intel / article

SLMs as Multi-Agent Routers: A Progressive SFT and Reinforcement Learning Approach

Source: arxiv.org
Author: Gayathri V Kondapalli, Alexander Ng, Hirsh Pithadia, Rahul Monish, Harvey Yorke, Amir Kayhani
Date: Published Aug 4, 2026

Context

The search that sounds most relevant to a question may return poor results. This study trains a small to choose among retrieval agents using feedback about the relevance of what they retrieve, alongside topic matching. It also learns to produce the structured parameters needed to call the chosen agent. Method.

The authors report better retrieval ranking than two models that selected agents from intent alone, including on a subset deliberately chosen for misleading topic matches. Average agent-selection time was 120.1 milliseconds; that measures the routing decision, not the time needed to retrieve and return an answer. The supported improvement comes from teaching the router which agents actually work well for different queries. Results.

Key takeaways · AI-distilled

Intent-based agent routing has a blind spot: it cannot detect when a topically aligned retrieval agent returns low-relevance results, because it never sees the retrieved content.
The fix: train a small LM with SFT then RL to jointly select the agent AND generate structured tool-call parameters, rewarded hierarchically on retrieval relevance plus topic .
On deliberately mismatched agent-query cases the trained SLM hits NDCG@10 of 0.918 vs 0.539/0.490 for Nova Lite and Claude Haiku 4.5 routing on intent alone.
Overall it wins on both quality and cost: mean NDCG@10 0.771 (+0.18-0.22 over the LLM baselines) at 120ms mean selection latency - an 82% latency reduction.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.

Key quotes

“Specialised retrieval agents typically surface higher quality results than general-purpose search, but selecting the optimal agent for a given query remains an open problem.”

“On a targeted subset of such agent-query mismatches, the trained model achieves an NDCG@10 of 0.918 compared to 0.539 and 0.490 for two LLM baselines (Amazon Nova Lite and Claude Haiku 4.5) that route on intent alone.”

Read the source arxiv.org