Cua's composite agents split desktop automation between a model that plans the task and one that locates the controls needed to carry it out. Planning decides what should happen next; visual groundingTying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.Full definition → translates that intention into a target on the screen. The framework announcement describes how developers can select the two models as one configuration.
The framework also translates differences in model outputs, such as whether a click location uses pixels or relative coordinates. This lets developers experiment with different planning and targeting models without writing a separate output parser for each supported combination.
Terms in this piece · Glossary
grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
