The First Few Tokens Matter: Understanding Chain-of-Thought Controllability
Source
arxiv.org
Author
Hugh Van Deventer, Dmitrii Troitskii, Laura Ruis
Date
Why it matters
If reasoning-trace monitoring is part of your AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → oversight, this shows models' limited control over their traces can be undone by prefilling the first tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → or banning common openers, so monitoring assumptions need testing.
Key takeaways · AI-distilled
Across models, scores correlate positively within the addition/style group and within suppression, while correlations across the two groups are mostly negative, so the authors treat them as separate capabilities.
The authors find the link to monitorability depends on which evaluation is used: addition and style controllability tend to track monitorability positively, while suppression controllability is often negatively associated with it.
The authors say their results argue for controllability evaluations that separate difficulty starting with a specific opening from broader limits on altering reasoning content, and that report control tasks and categories separately.
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
