How I tricked Claude into leaking your deepest, darkest secrets
- Source
- simonwillison.net
- Author
- Simon Willison
- Date
Context
A browsing tool can leak information even when it only follows permitted links. A July 9 demonstration showed Claude following a website's fake identity-verification instructions and revealing a user's name, employer and home city through its navigation choices. Each destination appeared on a previously fetched page, so the requests passed the tool's link restrictions while carrying private information to the site's operator.
The researcher reports that Anthropic closed this route by preventing the fetching tool from following links found inside fetched pages. The failure illustrates why restricting destinations is insufficient when an untrusted website can influence which permitted destination an assistant chooses.
- Claude's web_fetch was restricted to exact URLs the user typed or that web_search returned. That deterministically kills the classic attack of appending stolen data to evil.example.com/log?answers= and visiting it.
- The hole was that web_fetch would also follow links found inside pages it had already fetched. One attacker-controlled page then becomes an unlimited supply of pre-approved URLs.
- The exploit smuggled data out in the URL path. A fake Cloudflare "AI authentication" page told the to browse profiles letter by letter, so each fetch of coffee.evil.com/<letter> leaked one character.
- The page served the attack only to requests carrying the Claude-User agent string, so a human opening the same URL saw nothing suspicious. It extracted the user's name, home city, and employer.
- Anthropic's remedy was to delete the capability, not to filter it: web_fetch can no longer navigate to links returned inside its own fetched content.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
“Anthropic's protection is that web_fetch can only be used to navigate to exact URLs that the user has entered themselves or that were returned from its companion web_search tool.”
“web_fetch was also allowed to visit URLs embedded in pages that it had previously fetched, which meant you could create a honeypot site which encouraged the agent to exfiltrate data by following a sequence of nested generated links.”
“They were able to extract the user's name, home location city and the name of their employer.”
“Anthropic didn't pay out a bug bounty because they claimed to have identified it internally already, and have since closed the hole by removing the ability for web_fetch to navigate to additional links returned within its own fetched content.”
Checking sign-in…
Loading comments…