Using LLMs to Secure Source Code

Anthropic says the bottleneck in AI-assisted security work has shifted toward checking and fixing the vulnerabilities models report. Its source-code security guide separates discovery from verification: a fresh agent receives the finding and code, then looks for protections or other reasons the suspected exploit would fail. Keeping that check independent reduces the chance that it simply accepts the original agent's reasoning.

A verified finding still needs a tested repair. The guide recommends reproducing the bug, checking that the patch stops it without breaking other behavior, and searching for related vulnerabilities. The budgeting implication is concrete: increasing scan volume also requires capacity to verify findings, prioritize them and finish repairs.

“To give some indication of this discrepancy, as part of our own scanning of open source software, as of May 22, 2026, we had disclosed 1,596 vulnerabilities. To our knowledge, 97 of these have been patched.”

Eugene Yan

“Our primary takeaway: discovery is now straightforward to parallelize, and the bottleneck has shifted to verification, triage, and patching .”

Eugene Yan

“We’ve found frontier models to benefit from increasingly simple prompts during the discovery phase. Counterintuitively, more prescriptive prompts make discovery worse—long checklists tend to reduce the model’s creativity and generate fewer novel bugs.”

Eugene Yan

“Across the teams we’ve worked with, adding an adversarial verifier roughly halved the rate of non-exploitable findings from the discovery phase. Requiring that verifier to also build a proof of concept confirming the exploit brought the false positive rate to near zero.”

Eugene Yan

“Previously, when discovery took more effort, the engineer who found the bug also triaged it. Now, with models capable of finding a hundred candidates before lunch, triage has become the bottleneck.”

Eugene Yan