Intel / article

Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction

Source: arxiv.org
Author: Zhiqi Wang, Yichi Zhang, Dongwon Lee, Yuchen Yang
Date: Published Aug 13, 2026

Context

Shortening an AI assistant's conversation history can discard instructions that were meant to remain binding. COMPINT tests this problem by inserting continuing user constraints into long conversations and tasks, then checking what survives . The researchers report that the tested compactors retained only 17% of those constraints on average.

Their proposed fix extracts the continuing constraints separately, alongside the ordinary compactor. It achieved over 90% retention across the three tested scenarios without changing the underlying model or compactor. The result concerns preservation of instructions in these tests; it does not measure whether an agent subsequently obeys every preserved instruction.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context compaction — Summarizing an agent's earlier conversation to free room in the context window so a long session can keep going.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Key quotes

“We identify a class of user-issued instructions, Session Constraints (SCs), such as "do not delete any emails until I confirm," that are meant to constrain LLM's behavior for the remainder of a session but are silently dropped during compaction.”
Zhiqi Wang, Yichi Zhang, Dongwon Lee, Yuchen Yang

“To quantify this loss, we introduce COMPINT, an evaluation suite that evaluates compactors across three long-context scenarios: multi-turn chat, agentic trajectory, and long-horizon research.”
Zhiqi Wang, Yichi Zhang, Dongwon Lee, Yuchen Yang

“Current compactors retain only 17% of injected SCs on average, and most perform worse than running the same task without compaction.”
Zhiqi Wang, Yichi Zhang, Dongwon Lee, Yuchen Yang

“Retention varies sharply with compactor, prompt, context length, SC phrasing, and injection location, showing that the loss is systematic rather than tied to any single setting.”
Zhiqi Wang, Yichi Zhang, Dongwon Lee, Yuchen Yang

“We propose an SC-aware extractor that runs alongside the compactor as a plug-and-play module, achieving over 90% retention across all three scenarios without modifying the compactor or LLM.”
Zhiqi Wang, Yichi Zhang, Dongwon Lee, Yuchen Yang

Read the source arxiv.org