Question Decomposition Improves The Faithfulness Of Model Generated Reasoning
Source
anthropic.com
Author
Anthropic research
Date
Why it matters
If you rely on reasoning traces to audit AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → behavior, this shows that answering subquestions in isolated contexts makes the stated reasoning more faithful than chain-of-thought, at some cost in accuracy.
Key takeaways · AI-distilled
chain-of-thoughtHaving a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.Full definition → helps oversight only if the stated reasoning reflects the model's actual reasoning, which the authors note is not always the case.
Having models decompose a question into subquestions and answer each in a separate context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → greatly increased the faithfulness of stated reasoning compared with chain-of-thought on several recently proposed metrics.
The tradeoff is accuracy: decomposition kept some of chain-of-thought's question-answering gains, sometimes approaching its performance, while making the reasoning more faithful.
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
