Vibeleaderboard
← All Intel
Intel / post

Cua Previews Background Agent Input Isolation on Wayland

Source
x.com
Date
Cua@trycua
Thread · 5 parts

1/ At yesterday's Omarchy SF Meetup, we demoed a technical preview: an agent working across apps in the background with true multi-synthetic pointer. We'd like to work with the Hyprland community on native support for isolated background input. Here's what we have in mind.

2/ Here's @francedot demoing it live at the Omarchy SF Meetup at Cloudflare's SF office - running directly from Cua Fleet: https://t.co/8TodS4I5Ku

3/ We tested two compositor-side isolation paths: deliver directly to the target window's wl_pointer resource, or give automation a separate wl_seat. Both keep agent input away from the user's pointer-focus state and preserve the foreground drag.

4/ Both compositor approaches kept a real foreground drag intact during background delivery. As a next step, we're going to wire this in Cua Driver end to end, rebuild the current keyboard branch, resolve Chromium/Wayland behavior, and finish EIS device creation in the portal/libei path.

Read the full thread on X

Context

A computer-use that clicks and drags inside an app usually has to fight for the same pointer the person at the keyboard is using, which is why running an agent 'in the background' while someone keeps working is a genuinely hard problem. In a thread about a technical preview shown at an Omarchy meetup, Cua laid out two ways it tried to solve that on Hyprland, the Wayland compositor Omarchy uses.

The first path delivers synthetic input directly to the target window's own wl_pointer resource, the low-level object Wayland uses to track where a pointer is over a given window. The second gives the agent's automation a separate wl_seat, effectively a second virtual input device, so its clicks never touch the user's own pointer-focus state. Cua reports that both approaches kept a real foreground drag intact while the background input was being delivered, demoed live running from Cua Fleet.

This is explicitly a technical preview, not a shipped feature. Cua's stated next steps are finishing keyboard support, resolving Chromium-specific Wayland behavior, and completing device creation through the libei/portal path, followed by an RFC that would define the permissions and threat model before proposing the change to compositor projects.

Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
More from Cua
Recommended reads
Comments

Checking sign-in…

Loading comments…