An open Add/Search evaluation framework for agent memory
Source
agentmemoryleaderboard.ai
Author
IreneAI
Date
Context
The Agent Memory Leaderboard tries to isolate how well an AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → stores and retrieves information from how well its LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → answers questions. Entrants provide two interfaces: Add stores the supplied material, and Search returns relevant memories for a question. The platform then generates and scores answers using a fixed model, prompts and evaluation procedure.
Keeping that answering stage constant makes differences in the memory systems easier to interpret. Teams can choose their own storage and retrieval architecture, but cannot improve their score simply by substituting a stronger final-answer model. The resulting comparison applies to the platform's specified tasks and evaluation conditions.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
