Anthropic's October 2024 computer-use beta gave Claude 3.5 Sonnet a way to operate software through its visible interface. Developers could connect screen observations with commands to move the cursor, click and type, allowing an instruction such as filling a form to become a sequence of computer actions. This extended automation to ordinary software interfaces. The launch article described the capability as experimental and identified scrolling, dragging and zooming as difficulties, making it an early developer capability rather than evidence of reliable general desktop automation.
Terms in this piece · Glossary
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
