AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Why it matters
Routing models inside the agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition → instead of at the gateway cut aggregate inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → cost 58% and median turn latency from 81s to 49s in production, with task-completion and failure measures holding steady.
