How Much Memory Does Your Agent Actually Need?
- Source
- huggingface.co
- Date
Context
IBM's ALTK-Evolve experiments show why an 's accumulated guidance needs testing for each model. The system extracts reusable lessons from earlier task attempts and supplies them during later work without updating model weights. It compares sending every lesson with sending a fixed core plus lessons selected for the task. Method and results.
For gpt-oss-120b, selective delivery raised task completion from 39.9% to 56.0% on AppWorld's normal test split, while average use rose about 5%. The full guideline set gained less and used about 51% more tokens than no memory. Other tested models favored different amounts of guidance, making the finding a reason to calibrate memory delivery to the model and workload. Results and token-use tables.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
Checking sign-in…
Loading comments…