A computer-use AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → can fail a task and still leave damaging changes behind. LegacyWorld evaluates that distinction in older enterprise software operated through graphical interfaces. Its criterion, atomicity, requires either correct completion or failure without unintended persistent changes.
The researchers constructed 28 Windows workflows with defined starting states, goals and task-specific checks, then evaluated six hosted agents. They report different patterns of successful completion, safe failure and failures that leave changes behind. This makes the state of the records after an unsuccessful run part of the acceptance test, rather than treating every failure as equivalent.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.