If you're wiring an LLM into code review or a security-scanning agent, this gives you measured recall/precision and cost-per-run tradeoffs across models on real vulnerability discovery, plus the sobering baseline that even the best model finds only about a third of known issues — so you size human review accordingly instead of trusting the scanner.
A benchmark from Vercel that measures how accurately different LLMs (GPT-5.6, Claude Opus 5, Kimi K3, Grok 4.5, etc.) find real security vulnerabilities in application code, using a secret, human-judged golden set of 231 findings across 50 entry-point files to prevent memorization.
Each model is scored on a recall-weighted F2 metric alongside cost and runtime, revealing that even the best model catches only about 31% of known vulnerabilities while cheaper open-weight models increasingly rival frontier models on cost-efficiency.
