Adapting to AI penetration testing orchestrators: The Modern Security Model

Cloudflare says AI-driven attack tools went from one open-source project to dozens of competing families in about a month, and the break-ins it investigated succeeded through an old weakness: a long-lived API token with broad access. The tools are what Cloudflare calls penetration-testing orchestrators. Penetration testing is the authorized practice of attacking a system to find its holes. An orchestrator hands the stages of that work to several AI agents at once: they map a target's systems, probe what is exposed, try exploits and feed each confirmed result into the next step, with little human direction. The same software works against systems its operator has no permission to touch.

The project at the center is ARTEX, an open-source agentic penetration-testing system whose creator took it down on October 8, saying its malicious use ran counter to its purpose. In a post by Cloudflare's security researchers, the company reports that the takedown did not contain it. It counts over 12,000 repositories, several hundred forks that descend from the original, more than 50 framework families scanning Cloudflare customers in a three-day window, and daily requests from these tools up more than 30 times since October 1. As of November 12, ARTEX and its forks made up over 60% of the orchestrators Cloudflare knows about. Cloudflare suggests the takedown and the reporting on it likely caused a burst of forking, based on how sharply new instances rose around that date. That is the company's reading of the timing, not a demonstrated cause, and all of these figures are Cloudflare's own measurements from its network, which the post does not explain how it attributes to a particular framework.

Cloudflare's comparison of the five most popular tools shows how they divide the work. ARTEX runs a planning agent alongside parallel worker agents and tracks goals, leads and findings in a graph linked to a separate map of the target's assets. Others split the work between a supervisor and executor agents or dispatch specialist sub-agents, two of them run inside a Kali Linux container (a system packed with standard hacking tools) with root access, and one, Cairn, hands tasks to existing coding agents such as Codex and Claude Code. Cloudflare's argument is that this structure turns a break-in into a problem an agent can climb step by step: each verified finding narrows the next move, many agents work in parallel, and one operator can aim them at several organizations at once. Systematic testing of every exposed service also means small configuration mistakes get found.

The incidents show where that speed pays off for an attacker. Cloudforce One, Cloudflare's threat research and incident response team, investigated several cases in which an attacker got into a customer's account at a large cloud provider and moved from there into the customer's Cloudflare environment. The recurring root cause was static, non-expiring tokens used in CI/CD pipelines (the automated systems that build and deploy code) that granted broad access across environments. In one case, ARTEX used a compromised account-level token, previously used for CI/CD, that had no IP restrictions or rate limits. It created a large number of Workers, Cloudflare's small serverless scripts, named after their tasks with prefixes such as artex-pgrecon, and generated more than 69 requests per minute across several parts of the customer's Cloudflare setup and 400 database dump requests in an hour, along with attempts at persistence. Cloudflare believes a human operator lightly guided much of this. Its view of ARTEX misuse is not the only one: Wiz's roundup of agentic attacks cites Zscaler and CrowdStrike reports that attackers used ARTEX against South Korean financial institutions and took personal data on about 67,000 people.

Cloudflare's answer is a six-part "Modern Security Model": a web application firewall with rate limits, anomaly detection, bot defense, identity-gated access with micro-segmentation (dividing a network so one stolen credential cannot reach everything), strict validation of API requests, and threat intelligence. Each part links to a Cloudflare product, so it reads as vendor guidance as much as analysis. The evidence in the post points most directly at the credentials. The tokens behind these incidents were static and never expired, and the one in the ARTEX case worked across the whole account with no IP restriction or rate limit. Each of those properties is a setting its owner controls. Faster, automated attackers make that kind of forgotten key more likely to be found and used before anyone notices.