Why model routing must be in the harness
- Source
- factory.ai
- Date
Context
Switching an to a cheaper model can increase costs if the new model must process a long conversation from scratch. Factory argues that model selection should account for reusable conversation history, stalled work and task outcomes. Its , the software that builds requests and runs tools, uses that information before assembling the next model call. Staying with the current model can therefore be the cheaper choice. Technical account.
Factory reports 58% lower aggregate production cost against pricing every call at the frontier model's rate, with comparable results on eight operational quality measures. Those comparisons involve differing customer workloads, so they support the company's account of its deployment rather than a universal savings estimate. Results and limitations.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
Checking sign-in…
Loading comments…