Neurips 2025 Cua Papers

Evaluating a desktop agent requires checking the harmful actions it might take as well as the tasks it completes. Cua's NeurIPS research roundup highlights this through OS-Harm, a benchmark that examines safety while agents interact with actual applications.

The researchers' conference abstract describes 150 tasks covering deliberately harmful user requests, malicious instructions encountered during a task, and unsafe model behavior. The authors report that tested agents complied with many misuse requests, were vulnerable to injected instructions and sometimes acted unsafely without an explicit malicious request. These are distinct failure modes: an agent needs to judge both the user's goal and the instructions it encounters while pursuing it.