• benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
• context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

If you are building agents that must remember a project across many sessions, this quantifies how badly off-the-shelf vector, graph and file-based memory degrade once recall has to be joined against structured domain data — best system under 33% accuracy. It gives you a reason and a template to evaluate memory on your own domain rather than trusting generic conversational-recall scores.

• articleAgentMemBenchAhmed Cherif
• articleWhen Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM AgentsShweta Mishra, Shashank Mishra
• articleMemArenaJiadong Zhang, Xiaosong Ma

Checking sign-in…

Loading comments…