The Spatial Harness: Bringing Agents to the Canvas — Max Drake, tldraw
Source
youtube.com
Author
AI Engineer
Date
Context
An AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → working on a drawing canvas needs both a view of the picture and a way to act on its objects. The tldraw talk describes supplying a screenshot alongside the canvas's structured JSON, then extending the agent with its own task list and control over its viewport. Moving that viewport lets it look elsewhere on the canvas as work continues. The design addresses a concrete interface problem: useful spatial work requires access to the visual arrangement, the underlying objects and the area the agent is currently inspecting.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
videoCan Your AI Analyst Keep a Secret? - Aaron Linsky, Bridgewater & Nikhil Benesch, turbopuffer
videoWhat Does an AI Engineer Do Now? - Aparna Dhinakaran, Arize AI
videoDo Not Let Models Invent Numbers - Vinoo Ganesh, Kepler
videoWhy Frontier Models Fail as Buy-Side Analysts - Janie Gu, Google DeepMind & Suyeol Yun, LinqAlpha
