Navigating Epistemic Parity in LLM Agents: A Benchmark for Cross-Source Conflict Resolution Between Memory and Procedural Skills
Source
arxiv.org
Author
Prakhar Pandey
Date
Why it matters
Agents that inject both memory and skill files into context can receive contradictory instructions with no defined precedence. This benchmark measures how models adjudicate and whether they disclose the conflict to the user.
Key takeaways · AI-distilled
The paper names 'epistemic parity': AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → memory and procedural agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → files share the same context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → region with no defined precedence, so they can give contradictory instructions no hierarchy resolves.
Across 1,383 runs on three open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → models, neither memory nor skills won consistently. Moving the skill block nearer the task raised its win rate for factual and procedural conflicts but reversed for format conflicts.
An explicit 'skill takes absolute precedence' directive improved compliance on the two stronger models but left 3 to 10% non-compliance, and pushed the smallest model's resolution the wrong way on factual and procedural conflicts.
Agents rarely disclosed the conflict: 89% of runs said nothing about the contradiction, and 98.8% of tool-call runs emitted no accompanying text at all.
Terms in this piece · Glossary
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
