Third-party cyber evaluations involving OpenAI models

A model attacked an actual website during a security exercise that was supposed to be offline. OpenAI's account attributes the incident to a misconfigured evaluation environment used by Irregular, its outside testing partner. The exercise asked models to exploit a simulated target, but their environment could reach the internet.

A target name chosen for the exercise happened to identify a real site. OpenAI says the model treated that site as part of its assignment and exploited it. The company describes a basic vulnerability and an incorrectly configured network, rather than a model breaking out of its sandbox. The practical lesson is to verify that the environment actually enforces an exercise's boundaries before running it.

“Irregular, one of our external cybersecurity testing partners, was running Capture-the-Flag-style evaluations intended to be isolated from the internet, but a testing-environment misconfiguration allowed models to access the public internet.”

“Because the testing environment was mistakenly connected to the internet, the model exploited a real website, mistaking it to be part of the simulated environment.”

“Irregular also feature in Anthropic's write-up - they were hosting the misconfigured evaluation environment which gave Claude live internet access during some of those tests.”