UI-TARS Desktop
github.com/bytedance/ui-tars-desktop- Category
- AI Agents
- Rank
- No. 100Tools index
Previous survey · No. 120 ·
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Workflow Automation
- Interfaces
- Desktop · CLI
- Builder
- bytedance
- GitHub
- 39.1k stars
- Latest release
- v0.3.0
- Date
About
Open-source multimodal AI agent stack that lets you control computers and browsers through natural language commands. Connects cutting-edge AI models with real-world tools through desktop applications and CLI interfaces.
What it does
UI-TARS Desktop turns a vision-language model into an operator for graphical interfaces. Its desktop app can act on local or remote computers and browsers, while an accompanying SDK exposes the same GUI automation foundation to developers.
Why it's ranked here
The project covers more than a polished desktop shell. It includes reusable agent, browser automation, action parsing, event visualization, and MCP infrastructure. That breadth makes it credible for both direct use and custom automation, though the repository also combines a separate Agent TARS product that can blur the boundary.
What's good
The architecture exposes useful pieces instead of trapping automation inside the app. Developers get a typed GUI agent SDK, local and remote browser abstractions, and MCP connections over standard input, server-sent events, streaming HTTP, or memory. The visualizer processes uploaded execution dumps locally and provides timeline and replay views.
Tradeoffs
The repository is a sizable private pnpm monorepo requiring Node 20 or newer, so source development carries more setup than a standalone desktop download. Product documentation interleaves UI-TARS Desktop with Agent TARS, making capabilities easy to conflate. Telemetry delivery silently ignores failures, which limits diagnosis when collection breaks.
How to use it well
Pick it for teams experimenting with model-driven GUI automation across desktop and browser targets, especially when execution replay or a reusable SDK matters. Start with the desktop operator, then adopt the SDK for custom agents. Use the separately presented Agent TARS when the primary need is headless execution, a Web UI, or broader MCP workflows.
Technical notes+
package.json defines a private pnpm 9.10.0 and Turbo monorepo with Node >=20.x, TypeScript, Vitest, Playwright, Electron tooling, linting, formatting, coverage, and package release scripts. packages/ui-tars/sdk/src/index.ts exports GUIAgent, configuration and data types, status values, and model versions. packages/agent-infra/browser/src/index.ts exposes browser discovery plus local, remote, and base browser classes backed by puppeteer-core. packages/agent-infra/mcp-client/src/index.ts implements MCP clients for stdio, SSE, streamable HTTP, and in-memory transports, with allow and block filters and platform-specific executable paths. packages/ui-tars/visualizer/src/index.tsx renders uploaded execution dumps, replay controls, timelines, and detail panels; its uploader states that processing remains local. packages/ui-tars/utio/src/index.ts posts JSON events to a configured endpoint and silently catches failures.
Observed
- License
- Core cited packages carry Apache-2.0 SPDX headers; the visualizer source carries an MIT SPDX header.
- Primary language
- TypeScript and TSX, with small JavaScript executable entry points.
- Packaging
- Private pnpm 9.10.0 monorepo orchestrated with Turbo; source development requires Node 20 or newer.
- Interfaces
- Desktop application, GUI agent SDK, UI-TARS CLI package, browser automation library, and MCP client infrastructure.
- Operator targets
- Local and remote computer operators plus local and remote browser operators.
- Platform handling
- MCP process setup contains explicit paths and executable handling for macOS, Linux, and Windows.
Read from README.md, package.json, packages/agent-infra/create-new-mcp/index.js, packages/ui-tars/cli/bin/index.js, packages/ui-tars/cli/src/index.ts, packages/ui-tars/sdk/src/index.ts, packages/ui-tars/utio/src/index.ts, packages/agent-infra/logger/src/index.ts, packages/agent-infra/shared/src/index.ts, packages/agent-infra/browser/src/index.ts, packages/ui-tars/visualizer/src/index.tsx, packages/common/electron-build/src/index.ts, packages/ui-tars/action-parser/src/index.ts, packages/agent-infra/mcp-client/src/index.ts, packages/agent-infra/mcp-shared/src/index.ts.
What it can do
Control desktop applications through natural language commands
Natural language instructions → Automated desktop application interactions
Automate web browser navigation and interactions
Natural language browser commands → Executed browser actions and page interactions
Execute terminal commands via natural language
Natural language descriptions of terminal tasks → Executed command line operations
Connect AI models to desktop tools and applications
AI model requests and desktop application targets → Integrated AI-driven application control
Process multimodal inputs for computer automation
Text, voice, or visual commands → Interpreted automation instructions
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.