• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
• context compaction — Summarizing an agent's earlier conversation to free room in the context window so a long session can keep going.
• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

For long-running coding agents input tokens now dominate spend: compacting on cache miss turns cache-read cost from quadratic to linear and cut one real trace by 55%, but only works if compaction is fast enough to stay invisible.

• articleFast Agentic Search
• articleCode Retrieval
• articleRelace Apply 3
• articleRelace Repos

• clipCompaction vs. routing: why compaction is about intelligence, not costAI Engineer
• clipAuto-compaction for long-running agentsAI Engineer
• postPrompt caching in LLMs, clearly explainedAvi Chawla

Checking sign-in…

Loading comments…