Model Routing Belongs In The Harness
- Source
- factory.com
- Author
- Factory News
- Date
Context
Choosing a cheaper model for the next step can increase the total bill if the switch loses reusable processing of the conversation history. Factory, maker of the Droid coding agent, argues that routing should therefore happen inside the , the software that assembles requests, runs tools, and tracks the task. Its architecture account explains how that layer can weigh cache reuse against signs that a stronger model is needed.
The harness also constructs model-specific instructions and editing tools before sending a request. Selecting the model there lets the request fit its recipient. This makes routing a decision about the ongoing task and its accumulated state, rather than just the advertised price of one call.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- model routing — Sending each request to a model chosen by the difficulty of the task, rather than using one model for everything.
Checking sign-in…
Loading comments…