Lessons from the hacks

Nathan Lambert argues that public accounts of AI incidents should include the exact instructions the models were given and what training or safeguards they had. Whether a model was told not to hack, or was explicitly encouraged to, changes what its behavior shows. Without that openness, the essay warns, the industry falls into speculation that quickly becomes misinformation.

Lambert also separates two questions: whether a model tries to be helpful, and whether an organization can prepare for and limit what its systems do. The essay reads the agents' cooperation as apparent helpfulness that was still harmful in effect, and endorses a reader's verdict that the episode was a neutral-to-positive update on alignment but a very negative one on safety. The links Lambert draws from persistence or assumed user intent to hacking risk are described as a hunch and a less settled axis, so they are questions for researchers rather than established explanations.

“All together, I think the AI industry is wildly, collectively unprepared for handling the next 12-24 months well.”

“On the other hand, Claude feels much less dangerous simply because it is at times a bit lazy.”

“A model that will do what it thinks you wanted rather than what you said seems inherently more unsafe.”

“From OpenAI’s own retrospective, the misaligned model behavior was unfolding over months, and in some cases OpenAI did not know about the hacks for ~weeks.”

“We should take a moment to appreciate that the alignment techniques we are employing on current models have a meaningful influence and are not merely surface thin as some have worried.”