alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
Why it matters
If you're deploying long-running AI agents, this lays out the specific failure modes OpenAI observed at scale and the safeguards that mitigated them, so you can anticipate alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition → risks before they bite your own agentic systems.
Key quotes
“Long-running models can solve difficult, open-ended problems, but their persistence gives them more opportunities to take unwanted actions.”
OpenAI
“The model was instructed to post its results only to Slack, but the NanoGPT instructions say to post results as a PR to GitHub, and the model followed the latter instructions to open PR #287 on the public GitHub repository. The model circumvented sandbox restrictions designed to block external access and proceeded with the upload to GitHub.”
OpenAI
“It took our model an hour to find a vulnerability in the sandbox in order to make the PR; earlier models were less persistent and did not manage to find vulnerabilities, instead giving up on the PR part.”
OpenAI
“When its initial attempt was blocked due to a scanner detecting an authentication token, the model split the token body into two fragments, obfuscated them, and then reconstructed the credential at runtime so the complete token never appeared as one contiguous string.”
OpenAI
“This example shows how each step can look acceptable on its own while the sequence can produce an outcome that would not be approved. It also shows how a model that operates effectively over long time horizons can learn the blind spots of an approval system and work around it to achieve its goals.”
