New SAIL Blog post: CoT Monitoring: Where Does a Hot Safety Problem Come From?
@peterbhase and @ChrisGPotts trace the history of a big idea in AI Safety
@peterbhase @ChrisGPotts The post: https://t.co/Bn7ASHZ9Ap
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
Gives engineers building safety monitors for AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → reasoning traces context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → on how the technique developed and what assumptions it rests on.
