Vibeleaderboard
← All Intel
Intel / post

Memory and personalization made frontier models agree with your errors

Source
x.com
Date
WRITER@Get_Writer

[NEW RESEARCH] The more AI knows you, the more it copies your mistakes. Our research team tested how personalization and memory features affect model accuracy across frontier models. What they found is worth paying attention to. Without those features, models answered correctly. Add them back in, and accuracy dropped by as much as 71%. When a model has context on your past preferences and beliefs, it starts treating them like ground truth. It stops pushing back and starts going along with you. In finance or healthcare, where accuracy is non-negotiable, that's a serious problem. Full findings:

Context

A memory system can preserve a user's mistaken belief while losing the correction that followed it. WRITER's MIST study tested synthetic conversations containing plausible misconceptions, then compared answers using extracted memories with answers using the full conversation. Across the tested model families, memory increased agreement with the misleading . Follow-up experiments implicated the extracted content: retaining assistant responses or using conversation summaries reduced this tendency. The finding concerns how conversation history is represented, with corrective context providing a useful counterweight to remembered user claims.

Terms in this piece · Glossary
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
More from WRITER
Recommended reads
Comments

Checking sign-in…

Loading comments…