Vibeleaderboard
← All Intel
Intel / post

Safety blocks hit defensive cyber work: CyberGym results and costs

Source
ArtificialAnlys
Date
ArtificialAnlys@ArtificialAnlys

Restricting offensive without blocking defensive is difficult, as finding and proving a vulnerability requires the same steps whether the goal is to exploit it or patch it On CyberGym-E2E-AA, a benchmark that measures cyber defense capabilities from discovery to patching on memory-safety tasks, some frontier intelligence models are safety blocked from responding on 85%+ of tasks. The good news is that some of the most capable models are also the most cost effective. With GPT-6 Luna or MiMo-V2.6-Pro, you can run ~100 bug hunts in a 1M+ line codebase for ~$20 - up to 100x cheaper per task than the next most capable model Grok 4.7.

Why it matters

Some frontier models refuse over 85% of defensive vulnerability tasks because discovery steps look identical to offense. Pick the model with this in mind: GPT-6 Luna or MiMo-V2.6-Pro run about 100 bug hunts for roughly $20.

More from ArtificialAnlys
Recommended reads
Comments

Checking sign-in…

Loading comments…