DeepSeek V4.1 Flash 1017 tops open models on cyber defense index
- Source
- x.com
- Date
DeepSeek V4.1 Flash 1017 is the new #1 openly-available model in the Artificial Analysis Cyber Index. Other proprietary models may offer better cyber defense performance but are restricted access @deepseek_ai recently released DeepSeek V4.1 Flash 1017, an update to their flagship open-weights model DeepSeek V4.1 Flash. With an Artificial Analysis Cyber Index score of 57%, DeepSeek V4.1 Flash 1017 is the #1 openly-available model for cyber defense capabilities. The Artificial Analysis Cyber Index measures capability on agentic cyber defense tasks and is composed of three benchmarks: CWE-Bench-AA, DeepsecBench-AA and CyberGym-E2E-AA. The Cyber Index primarily evaluates models through their publicly accessible APIs, reflecting the cyber capabilities available to developers. Many models, particularly proprietary models from OpenAI, Anthropic and Google, may have stronger defensive cyber capabilities but encounter safety refusals that reduce their overall scores. GPT-6 Sol (Daybreak Blue, max), available through OpenAI’s trusted-access program, leads the Index, scoring 32 points above its public counterpart driven by fewer refusals. DeepSeek V4.1 Flash 1017 encounters no safety…
A cheap open model reaches 57% on agentic cyber defense at $0.42 per task, about 4x cheaper than trusted-access GPT-6 Sol. Its 53-point CyberGym gain shows how fast open models are closing the gap.
- Artificial Analysis says DeepSeek V4.1 Flash 1017 (max) scores 57% on its Cyber Index, one point ahead of Grok 4.7 and MiMo-V2.6-Pro, the top openly available result. Only trusted-access GPT-6 Sol (Daybreak Blue, max) scores higher, at 69%.
- The 16-point gain over the prior DeepSeek V4.1 Flash came almost entirely from CyberGym-E2E-AA, up 53 points to 76%. CWE-Bench-AA barely moved (68% to 69%) and DeepsecBench-AA fell from 31% to 25%.
- Refusals shape the ranking: the index tests public APIs, DeepSeek's model hit no safety refusals, and GPT-6 Sol's trusted-access variant scores 32 points above its public counterpart. At $0.42 per task it costs 4x less than that variant.
Checking sign-in…
Loading comments…