Cua

Traces computer-use agents from OpenAI's 2016 Universe project through GPT-4V's vision capability and Karpathy's 'LLM OS' framing into recent viral agents, arguing vision-language models were the missing prerequisite the field needed.

Details the private macOS SkyLight internals (SLEventPostToPid, sparse accessibility SPIs, yabai's focus-without-raise trick) that cua-driver uses to let agents click and type inside background Mac apps without hijacking the user's cursor or Space.

Cua's own thread shows how to wire OpenAI's Agents API into a Cua Cloud Fleet VM: the agent loop stays on OpenAI's side while Cua Driver executes desktop actions locally over MCP, and a claimed sandbox can be disconnected and reattached mid-task.

Walks through the @trycua/computer Web SDK, which lets JavaScript/TypeScript apps drive Cua cloud sandboxes with OpenAI's computer-use-preview model, covering setup, the click-type-screenshot loop, and framework integration in Node, React, or Vue.

Cua's early-access walkthrough of Gemini 4 Argon driving native macOS apps in the background via Cua Driver. Covers MCP setup in Antigravity, SDK function declarations for custom agent loops, and running long jobs in isolated Cua Spaces. Demos are vendor-reported.

Cua released cua-driver SDKs for Python and TypeScript that load its Rust engine in-process, plus a background input mode that lets an agent click an element via getWindowState() without bringing the window forward.

Cua's AI Engineer World's Fair talk traces computer-use from a foreground screenshot-and-click loop to Computer-Use 2.0, where a coding agent calls desktop control as one tool among many, backed by Cua Driver, Cua-Bench, and Cua Fleets infrastructure.

Cua explains how to record human computer-interaction trajectories with its Computer-Use Interface and Gradio UI, then package and publish them to Hugging Face as training data for computer-use models.

Cua Cloud Fleets added a creation-age TTL setting for pools and claims, so temporary agent sandboxes used in CI or eval runs delete themselves automatically instead of needing manual cleanup.

Cua released Cua-S1-4B-0.2, a multimodal decision model trained with RLOO on live computer-use tasks using task-completion rewards, reaching 92.9% on a frozen GUI-360 benchmark split versus 60.1% for the untrained base model, with Apache-2.0 adapters.

Cua explains how jev-use turns screen state into bounded action candidates that a fast decision model (Jev) scores instead of calling a general LLM, how OmniParser fills the perception gap for screenshot-only UIs, and where the 'System 1/2' framing for small models breaks down.

Cua released Cua-Bench-S1, a benchmark for single-decision computer-use models across interface actions, safety abstention, and game control, plus held-out probes like chess and Doom, along with two open checkpoints: an 855K-parameter Cua-S1-Nano and a Qwen3.5-4B LoRA.

Cua Agent framework 0.4 normalizes coordinate formats and tool-call conventions across GUI models (Anthropic, OpenAI, Hugging Face) behind one model string, and introduces composite agents pairing a grounding model for clicks with a planning model for reasoning.

Analyzes Apple's WWDC Containerization framework, which runs each container in its own lightweight VM rather than sharing one big VM like Docker, arguing the design validates Cua's own Lume approach while hinting at coming GPU passthrough for local model inference.

Cua VLM Router gives one API key and an OpenAI-compatible endpoint for multiple computer-use model providers, starting with Anthropic's Claude Sonnet 4.5 and Haiku 4.5, auto-routing across Anthropic, Bedrock, or Microsoft Foundry for availability.

Cua introduces a browser-based Trajectory Viewer for stepping through a computer-use agent's saved observations, actions, and decisions, useful for debugging failures and exporting data for training.

Shows how Microsoft's built-in Windows Sandbox, a free disposable VM on every Windows PC, can serve as a lightweight local test environment for computer-use agents automating enterprise apps like AutoCAD or SAP before moving to cloud infrastructure.

Explains how cua-driver's Windows backend unifies window capture, UI Automation trees, and synthetic input across Win32, WPF, WinUI, and Electron, cataloging real failure modes: black DirectComposition frames, ignored PostMessage clicks, Session 0 blindness.

Cua published a guide for packaging the official OSWorld Ubuntu benchmark image with Cua Driver and running it as a Cua Fleet desktop, so agent builders can run and evaluate OSWorld tasks on Cua's infrastructure.

Introduces the @sandboxed decorator, which runs a full Python function inside the same container as an agent's UI automation, letting an agent click and type through the GUI while executing custom code and calling APIs in one isolated environment.

Cua ran Gemini 3.5 Flash's native Computer Use API against GPT-5.5, Claude Sonnet/Haiku/Opus, and Gemini 3.1 on 25 real KiCad EDA tasks. Flash won on mean reward via pixel-accurate grounding, but all models struggled with from-scratch design and screenshot-state hallucination.

Cursor Cloud Agent workers can now run on Windows and Linux via Cua Cloud Fleets, with Cursor driving the agent loop and a Cua VM executing tool calls, requiring Cursor Enterprise self-hosted agents.

Cua Playground lets you send tasks to a running cloud sandbox from the browser and watch an agent's tool calls and screenshots live, with Claude Sonnet 4.5/Haiku 4.5 support, aimed at fast prompt iteration without redeploying code.

Cua releases the Spaces SDK for giving Claude Code, Codex or any agent its own computer on hardware you choose, with jobs run across all your machines at once. An update extends it to your own Hetzner and AWS accounts and E2B/Daytona code, in early access.