Intel / article

Quoting huggingface.co/security.txt

Source: simonwillison.net
Date: Published Sep 11, 2026

Context

On September 11, Simon Willison preserved a message attributed to Hugging Face's security.txt asking AI agents seeking vulnerabilities to use the CyberGym instead. The message proposes a designated place for testing security capabilities rather than probing the live service.

CyberGym's documentation gives that alternative concrete substance: it supplies vulnerability-analysis tasks and a system for checking submitted proof-of-concept inputs. It instructs operators to deploy locally and keep the service off the public internet. The archived message is a request to agents; the benchmark's deployment controls address where their testing actually runs.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Read the source simonwillison.net