Teaching AI to Find Real Vulnerabilities — David Brumley, Bugcrowd

An AI security evaluator can reward finding one easy flaw repeatedly while missing whether the system can conduct a broader audit. Bugcrowd's presentation proposes asking for multiple vulnerability findings and requiring reproducible evidence for each. The grader checks the submitted evidence, distinguishes findings and balances valid discoveries against unsupported submissions. It can also recognize a demonstrated flaw that was absent from the initial answer set. This changes the incentive from obtaining any passing result to producing a useful collection of validated findings. The success condition still needs to match the task: causing a program to crash establishes a different capability from demonstrating control over it.

“hacking is really a ladder. And this is what actually matches cybersecurity so well to reinforcement learning.”

David Brumley

“Crashing a program is different than hacking it. You can't go steal someone's IP by simply crashing a program.”

David Brumley

“don't define the task by a single bed, let the program define the task”

David Brumley

“if you could give Chrome to an LLM and it could come up with a zero-day, you would essentially be able to hack nation-states at that point”

David Brumley

“you can't judge trust the LLM that you're teaching to be a judge”

David Brumley