If you run agents over long sessions, compaction strategy silently determines what your agent forgets — this lays out a repeatable way to measure which summarization approach preserves task-relevant context, with side-by-side results against OpenAI and Anthropic's compaction behavior.
Read the methodology as the takeaway and treat the winner claim as vendor-flavored.
Factory.ai built an evaluation framework to measure how well different context-compression strategies preserve useful information during long-running agent sessions (debugging, code review, feature implementation).
Testing across real-world tasks, they found structured summarization retains more relevant context than alternative approaches from OpenAI and Anthropic.
Transcript
We built an evaluation framework to measure how much context different compression strategies preserve. After testing three approaches on real-world, long-running agent sessions spanning debugging, code review, and feature implementation, we found that structured summarization retains more useful information than alternatives from OpenAI and Anthropic.
