Framework and Benchmark for Code-Driven Agentic Testing in Web Development
- Source
- arxiv.org
- Author
- Bin Hong, Zhenchao Zhang, Jiyuan He, Kai Zhang, Zhenya Huang
- Date
Context
An AI tester can recognize that a web application is faulty while missing the defects that need fixing. CATTest measures those abilities separately across 102 AI-generated web applications with annotated bugs. Its testing writes Playwright browser-automation code, examines feedback and explores the application to produce bug reports.
The strongest reported bug-discovery score was 42.57 out of 100, calculated from per-project coverage of annotated defects. Some models scored much better at simply flagging faulty applications. That gap explains why a convincing overall verdict is insufficient evidence of thorough testing: developers need reports that identify the particular failures they can reproduce and repair.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleWebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution HarnessYun-Yun Tsai, Yuning Mao, Shiqi Wang, Junfeng Yang, Sinong Wang
articleGeneration of Web Apps with Agentic IDEs: An Empirical AssessmentManuel Marceca, Maria Teresa Rossi, Leonardo Mariani
articleHxAgent: Iterative Agent Planning for End-to-End Web Application TestingTu Nguyen, Duy Cao, Viet Nguyen, Phu Nguyen, Vy Le, Nguyen TK Nguyen, Tien N. Nguyen, Vu Nguyen
Checking sign-in…
Loading comments…