Intel / article

Context Poisoning as Extreme-Value Attention Interference in Long-Context Language Models

Source: arxiv.org
Author: Meysam Ghaffari, Nina Fatehi, Bhaskar Sen, Nasim Sabetpour, Carlos Morato
Date: Published Sep 22, 2026

Context

Giving a more can make decisive evidence harder to identify when the added material contains convincing distractions. This study models the problem as competition between relevant evidence and distractors: increasing the number of distracting candidates increases the chance that one receives a misleadingly strong score. The authors report that, at a fixed context length, distractors with the same format as the evidence caused the largest accuracy drop among the constructions tested. Filtering context could improve evidence use, but its benefit depended on retaining the evidence that mattered. The finding makes relevance and resemblance part of the context problem, alongside length.

Key takeaways · AI-distilled

The paper models context poisoning as extreme-value interference: the decisive evidence's score is bounded, while the maximum score among distractors keeps growing as more are added.
The N that matters is the effective count of confusable distractors, not raw context length, and the analysis links degradation to score aliasing, positional aliasing and softmax dilution.
In controlled tests, distractors in the same format as the evidence caused the largest accuracy drop among the tested distractor types at fixed context length.
Retrieval gating improved evidence use, but its net benefit depended on preserving evidence recall; the authors point to evidence bottlenecks, retrieve-then-reason designs and verifier-mediated memory as mitigations.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.

Read the source arxiv.org