- Category
- AI Agents
- Rank
- No. 218Tools index
- Type
- APP
- Builder
- trycua
- GitHub
- 22.4k stars
- Date
About
Build, benchmark, and deploy agents that use computers
What it does
Cua supplies a common control layer for virtual machines, containers, and native desktops. Agents can inspect screens, run shell commands, click, type, and perform mobile gestures. Companion components provide background desktop control, repeatable evaluation environments, trajectory export, and Apple Silicon virtualization.
Why it's ranked here
The strongest case is coverage without separate control models for every operating system. One sandbox API spans Linux, macOS, Windows, and Android, while the driver exposes both command-line and MCP interfaces. Benchmarks and trajectory export make the suite useful beyond demonstrations. The breadth also means adopters must choose among several related packages and installation paths.
What's good
Background desktop actions do not take over the user’s cursor or focus. The sandbox abstraction works across cloud and local runtimes, including custom local disk images. Evaluation support includes established task sets, custom tasks, parallel runs, and exported trajectories. The repository also separates tutorials, task guides, concepts, and reference material by reader intent.
Tradeoffs
Linux background input has explicit Wayland compositor limitations. Cloud use of custom disk images is still listed as forthcoming, although local QEMU accepts them. Lume targets Apple Silicon, and one macOS preset may expose an Accessibility setup step on first display boot. Installation is fragmented across Python packages, shell installers, PowerShell, and source installation for benchmarking.
How to use it well
Use the sandbox API when an agent needs disposable, cross-platform machines. Use the driver when an existing coding or custom agent must operate native applications without disrupting the foreground session. Add the benchmark component for evaluation, training environments, or trajectory collection. Treat Lume as the Apple Silicon virtualization layer, not as a general virtualization answer for every host architecture.
Technical notes+
The root pyproject.toml defines a uv monorepo with workspace members under libs/python/agent, libs/python/core, libs/python/computer, libs/python/computer-server, libs/python/som, libs/python/mcp-server, and libs/python/bench-ui; it requires Python 3.12 through 3.13, while README.md says the installed cua API requires Python 3.11 or later. Strict mypy settings, Ruff, Black, pytest groups, and async test configuration are present. package.json contributes pnpm-based Prettier checks rather than the primary runtime package. docs/scripts/crawl_docs.py, docs/scripts/generate_db.py, and docs/scripts/modal_app.py implement Playwright crawling, LanceDB embeddings, SQLite search, scheduled Modal jobs, and an MCP documentation service.
Observed
- License
- MIT; bundled Kasm is MIT, OmniParser is CC-BY-4.0, and the optional Omni agent extra includes an AGPL-3.0 dependency.
- Primary language
- Python is the main SDK and workspace surface; the repository also contains Rust virtualization and driver components plus TypeScript documentation tooling.
- Packaging and install
- The sandbox SDK installs from Python packaging; desktop drivers and Lume use platform shell installers; the benchmark tool supports editable installation with uv.
- Interfaces
- Python API, command-line tools, MCP server, and a Docker-compatible interface for Lume virtual machines.
- Platform support
- Sandbox targets Linux containers, Linux virtual machines, macOS, Windows, and Android in cloud and local QEMU configurations.
- Repository structure
- Monorepo containing agent, sandbox, computer server, benchmark, desktop driver, virtualization, documentation, and documentation-search components.
Read from README.md, Makefile, package.json, pyproject.toml, docs/README.md, docs/release-backfill-runbook.md, docs/release-attribution-and-announcements-plan.md, docs/scripts/README.md, docs/next.config.mjs, docs/source.config.ts, docs/postcss.config.mjs, docs/scripts/modal_app.py, docs/scripts/crawl_docs.py, docs/scripts/check-links.ts, docs/scripts/generate_db.py.
What it can do
Build computer-controlling agents
Agent specifications and configuration → Functional computer automation agents
Benchmark agent performance
Computer agents and test scenarios → Performance metrics and evaluation reports
Deploy agents to production environments
Built and tested agents → Running agent deployments
Automate computer tasks through agents
Task definitions and parameters → Executed computer operations and results
Package agents in Docker containers
Agent code and dependencies → Containerized agent applications
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.