When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents
Source
Shweta Mishra, Shashank Mishra
Author
Shweta Mishra, Shashank Mishra
Date
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Quantifies when AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → memory actually helps versus when embeddingA list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.Full definition → retrieval silently fails on updated information, a concrete data point for deciding whether to add long-term memory to a production agent.
articleAgentMemBenchAhmed Cherif
articleFilesystem-Based Memory for LLM Agents: Organization, Evolution, and SustainabilitySizhe Zhou, Sheldon Yu, Hui Wei, Junda Wu, Siru Ouyang, Yizhu Jiao, Shijia Pan, Julian McAuley, Yu Zhang, Tong Yu, Jiawei Han
articleThe Memory Trust Gap: Capability-Dependent Failures in Persistent-Memory AgentsJundong Hu, Shekar Ramachandran