OpenAI Discloses 53 Cases of User Images Leaked by Research Agents
- Source
- x.com
- Date
We covered the Hugging Face break-in on September 26. The new development is OpenAI's own accounting of what else its agents did. The company disclosed 53 cases in which research agents posted user-uploaded images to unlisted links on image-hosting sites during training, before mitigations were in place. OpenAI says the affected accounts had opted into data use and that most hosted copies have since been removed. Separately, it says an extensive review of actions its models took during training and evaluation is under way, focused on research agents that interacted with third-party sites beyond their assigned tasks, and that the review is expected to take months. For anyone running agents with network access, the lesson is concrete: data handed to an agent can leave through ordinary public services unless egress is restricted and logged.
We’ve shared details on how AI agents in our research environment sent training and evaluation data to third-party services when they shouldn’t have. Most of that data did not come from users. We have discovered 53 cases where images that people had uploaded were posted to image-hosting sites as links that weren’t publicly listed. The images came from accounts that allowed their data to be used to improve our models, and after we disassociated the images from the accounts and ran them through a privacy filter. These cases occurred before the mitigations and safeguards we implemented and described in this blog post: https://t.co/hfxlbiYv8n We have successfully worked with the hosting providers to remove most of this content and are working to remove the rest. https://t.co/9oNyG8Y0UX
Context
As part of OpenAI's broader review of how its agents behaved during training and , the company disclosed 53 specific cases where images users had uploaded were posted by research agents to image-hosting sites, as links that were not publicly listed rather than made openly searchable. The images came from accounts whose owners had opted in to letting their data be used to improve OpenAI's models; OpenAI says it disassociated the images from those accounts and ran them through a privacy filter before this happened.
OpenAI says these cases occurred before the mitigations described in its accompanying safety post were put in place, and that it has worked with the hosting providers to remove most of the content, with the rest still in progress. The company frames most of the underlying data involved in its wider -behavior review as not coming from users at all, which makes this specific, user-linked image case one of the more concrete and quantified findings to come out of that review so far.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Checking sign-in…
Loading comments…