Thinking of ACE? We Can Do It with Fewer Tokens
- Source
- huggingface.co
- Date
Context
IBM researchers' ALTK-Evolve comparison examines how much remembered guidance an needs. Both ALTK-Evolve and ACE turn an agent's previous attempts into reusable lessons. ACE supplies its comprehensive playbook at every step; ALTK-Evolve can retrieve a smaller selection relevant to the task.
In IBM's own AppWorld runs, which included running ACE in-house, selected guidance with gpt-oss-120b used 116,000 per task versus ACE's 777,000. Task completion was 56.0% versus 54.8%, which the authors treat as effectively tied because of run-to-run variation. For the stronger DeepSeek-V3.2 model, IBM delivered the full consolidated set instead, so how much to send is a per-model choice. The comparison used different prompt templates and measures token consumption, not a production invoice.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…