Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs
Source
arxiv.org
Author
Boris Radovi\v{c}, Marco Canini, Veljko Pejovi\'c
Date
Why it matters
You can keep private data on the device and still recover much of a cloud ReAct agent's accuracy. The cloud model writes code and a local small model answers the data-dependent questions.
Key takeaways · AI-distilled
Code-only agents fail because the cloud model writes the program before seeing any user data, so any step that depends on interpreting runtime content breaks. The hybrid defers those bounded questions to a local model.
The improvement held in all six combinations of backbone model and AppWorld test partition the authors evaluated, peaking at 14.9 percentage points over the code-only AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →.
It narrows but does not close the gap: the authors say it recovers a substantial part of the distance to a cloud-hosted ReAct loop that reads user data directly.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleWhy Better Models Can Create Riskier Systems: Evidence from LLM Agents in Financial MarketsJillian Ross, Eric So, Zoe De Simone, Charles Pozniak, Andrew W. Lo
articleCallability Is Not Operability: Controlled Interface Interventions for LLM AgentsZihao Wang
articleDo Agents Know When They Succeed? Calibrating Agent Confidence from Internal RepresentationsPriyanka Mary Mammen, Emil Joswin, Srujananjali Medicherla