Intel / article

CUADebug

Source: arxiv.org
Author: Weijia Zhang, Kunlun Zhu, Zeyi Liu, Yinting Chen, Tianyi Ma, Jiateng Liu, Jiaxun Zhang, Bingxuan Li, Xiangru Tang, Heng Ji, Jiaxuan You
Date: Published Aug 5, 2026

Context

A computer-using 's final failure may originate several actions earlier. CUADebug searches for that earlier mistake by inspecting selected steps, paired screenshots and action records. It returns a diagnosis identifying the causal step, supporting evidence and a correction to guide another attempt. Framework description.

In the revised study, a single retry with this approach recovered 29.86% of failed tasks, compared with 13.89% in the comparison condition. Human annotations fixed restart points in controlled retry comparisons, although the debugger produced its diagnoses without intervention on each trajectory. The result supports diagnosis-guided recovery under that setup. Revised results.

Key takeaways · AI-distilled

Human annotation of 204 failed OSWorld trajectories puts task reasoning and control as the largest computer-use-agent failure family (110/204), ahead of perception (36) and /interaction (25).
CUADebugger actively inspects suspicious steps with paired before/after screenshots instead of prompting once over the whole trajectory, emitting a structured root-cause diagnosis with corrective strategy.
Root-cause-informed re-execution nearly doubles task completion vs history-only continuation (29.9% vs 13.9% single-shot; 12.2% to 25.9% continual, near the 29.2% human-oracle ceiling).
The takeaway for agent builders: diagnosis is an actionable repair signal, not just a post-hoc explanation - and it transfers across debugger backbones.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.

Key quotes

“CUA failures arise from coupled visual perception, spatial grounding, low-level interaction, task reasoning, and environment dynamics, making debugging a distinctive multimodal causal localization problem.”

“These results show that CUA root-cause diagnosis can provide actionable repair signals rather than merely post-hoc explanations.”

Read the source arxiv.org