- Category
- AI Agents
- Rank
- No. 461Tools index
Previous survey · No. 455 ·
- Pricing
- Open Source
- Type
- AGENT
- Builder
- microsoft
- GitHub
- 9.7k stars
- Latest release
- v3.0.8
- Date
About
Microsoft Research's multi-agent Windows automation framework — UI-Focused agents (UFO3) that coordinate to operate Windows apps.
What it does
UFO offers two automation layers. UFO² handles one Windows machine through GUI controls and native interfaces. Galaxy breaks larger requests into dependency graphs, assigns work by device capability, runs independent tasks concurrently, and exchanges status and results over persistent WebSocket connections.
Why it's ranked here
The architecture addresses unusually broad automation work without abandoning the simpler desktop path. Dynamic task graphs, parallel scheduling, device matching, typed messages, reconnection, and MCP integration form a coherent system. The cost is meaningful setup and a larger operational surface, especially while Galaxy remains under active development.
What's good
Galaxy can revise task graphs from execution feedback instead of treating plans as fixed. Capability-based assignment supports Windows, Linux, and Android agents. UFO² combines GUI actions with Windows UI Automation, Win32, and COM controls. It can also batch predicted actions to reduce language-model calls.
Tradeoffs
Galaxy requires provider credentials, device registration, platform-specific agent setup, and multiple running services. Its documented learning curve and setup difficulty exceed UFO². Windows-native automation dependencies only install on Windows. The dependency surface is large, spanning language-model clients, vector search, web servers, WebSockets, MCP, and desktop-control packages.
How to use it well
Use Galaxy when a workflow has real dependencies, parallel branches, or work distributed across different device types. Start with UFO² for focused Windows application automation, then attach it as a Galaxy executor when coordination becomes necessary. This is not the economical choice for a simple single-machine task that needs no orchestration.
Technical notes+
galaxy/galaxy.py implements request, interactive, demo, mock, and web UI modes around GalaxyClient; it also exposes programmatic quick-start and interactive coroutines. galaxy/__main__.py supplies package execution, while ufo/__main__.py launches the desktop agent. aip/__init__.py exports typed protocol, endpoint, transport, heartbeat, timeout, and reconnection components; aip/messages.py uses Pydantic models for commands, results, controls, and client/server messages. requirements.txt pins most dependencies and gates PyPDF2, pywin32, pywinauto, pyautogui, uiautomation, and comtypes to Windows. learner/indexer.py builds FAISS indexes from application documentation and supports incremental merging. docs/superpowers/plans/2026-08-10-ipv6-transition-ssrf.md is an implementation plan, not evidence that its proposed URL-security tests or transition-address protections are already present.
Observed
- License
- MIT
- Primary language
- Python, with documented support for Python 3.10 and 3.11
- Install surface
- Dependencies are installed from requirements.txt with pip
- Interfaces
- Command-line modes, Python imports, a local FastAPI web UI, WebSocket messaging, and MCP integration
- Platform support
- Galaxy documents Windows, Linux, and Android device agents; UFO² targets Windows desktop automation
- Architecture
- Galaxy uses dependency-graph orchestration, while UFO² uses a sequential ReAct loop
- Configuration
- Both frameworks require language-model provider configuration
Read from README.md, requirements.txt, docs/superpowers/plans/2026-08-10-ipv6-transition-ssrf.md, aip/__init__.py, ufo/__init__.py, ufo/__main__.py, aip/messages.py, galaxy/galaxy.py, learner/basic.py, learner/utils.py, config/__init__.py, galaxy/__init__.py, galaxy/__main__.py, learner/indexer.py, learner/learner.py.
What it can do
Automate Windows application workflows
Natural language instructions describing desired tasks → Executed sequences of UI interactions across Windows applications
Coordinate multiple AI agents for complex tasks
Multi-step automation requirements → Orchestrated agent actions to complete workflows
Navigate and interact with Windows UI elements
Target application windows and UI components → Clicks, keystrokes, and other UI interactions
Parse and understand Windows application interfaces
Active Windows applications and their UI structures → Identified UI elements and interaction possibilities
Execute cross-application automation workflows
Tasks requiring multiple Windows applications → Coordinated actions across different software programs
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.