Managing a token budget? Think efficiency, not intelligence
- Source
- writer.com
- Author
- Writer Team
- Date
Context
Writer argues that an 's cost depends on how much work the software around the model repeats, not only on the model's price. Its budget analysis says to measure cost per completed task, including repeated model calls and retries. The agent's , the software managing prompts, tools and execution, controls much of that workload.
The company's engineering account describes keeping reusable prompt material in a stable prefix so it can be cached, and suspending jobs that are waiting on a person or a long task rather than repeatedly polling the model. Across six models and 22 prompts, Writer reports cost and duration cut by more than 40% at quality parity. The authors call the quality findings directional and say 22 prompts are too few for quality conclusions, so this supports testing workflow efficiency as a cost lever, not a general saving.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Checking sign-in…
Loading comments…