Vibeleaderboard
← All Intel
Intel / article

DVBench: End-to-End Benchmark for Real-World UVM Design Verification Agents

Source
arxiv.org
Author
Jiaru Qian, Jiawei Li, Xiaolong Hu, Zheng Shen, LiMing Zhao, Xin Zhang, Siyuan Jiang, Yongqiang Wei, Dong Liu, Jianyang Li, Yihua Chen, Yangxu Liao, Yuxin Wu, Jing Liang, Bin Wen, Junjie Zhang, Yongsheng Han, Yihong Dong, Ge Li
Date
Why it matters

Existing hardware verification benchmarks test decoupled subtasks; this one scores agents on the whole UVM workflow using compilation, coverage and bug detection, which is closer to real work.

Key takeaways · AI-distilled
  • DVBench has 95 tasks across 9 functional domains and 5 difficulty levels. Each requires an to complete the full UVM verification workflow on its own, unlike earlier benchmarks built from decoupled subtasks.
  • Grading covers the final testbench through compilation, coverage and bug detection, and also assesses intermediate artifacts: feature lists, test plans and reference models.
  • The best agent tested, Codex with GPT-6 Astra, reached only a 21.05% overall success rate. The also covered Claude Opus 5.5 and the Claude Code and OpenCode agent frameworks.
Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Recommended reads
Comments

Checking sign-in…

Loading comments…