How we make AI coding more cost efficient without sacrificing task quality
- Source
- github.blog
- Author
- Erik Kristensen
- Date
GitHub documented four changes intended to reduce the cost of Copilot’s coding agent without reducing task quality. The team trimmed repetitive tool output, removed formatting that consumed tokens without helping the model, shortened instructions and returned completed background work without an extra retrieval step. Each change was checked with offline benchmarks and then an A/B test, which matters because fewer tokens per call can still increase total cost if the agent needs another turn to recover missing context. The transferable lesson is to optimize cost per successful task rather than prompt size or request price in isolation. Teams running their own harnesses can apply the same measurement discipline even when their tools differ.
Context
GitHub found an example of information-free work in Copilot CLI: line-number prefixes attached to every line of every file the read. Its editing tools now locate changes through surrounding code, so those prefixes consumed model input without helping the normal editing process. Engineering account
Removing the prefixes cut average daily model- cost per user by about 3% in the company's online experiment with Copilot CLI users, with no material regression detected in the quality or satisfaction measures it tracked. File contents stayed intact. The example shows why cost reduction can come from changing what the surrounding software sends, without withholding source information from the model.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
Checking sign-in…
Loading comments…