Notion’s Sarah Sachs & Simon Last on Custom Agents, Evals, and the Future of Work
- Source
- youtube.com
- Author
- Latent Space
- Date
Context
Notion's Simon Last and Sarah Sachs recount in this Latent Space interview that the company tried to build an agent that could use all of Notion's tools as early as late 2022, when it first had access to GPT-4, and kept failing. Last says there was no common idea of yet, the models were not capable enough and windows were too short. The team designed its own tool-calling framework and partnered with Anthropic and OpenAI at different times to fine-tune models to use it over multiple turns.
Last says the unlock was a Claude Sonnet release recalled as 3.6 or 3.7, after which Notion built the agent it shipped. Custom agents, which run in the background, took longer because the team wanted far higher reliability and a permissions design for cases where a shared agent's audience and its document access differ. Sachs describes a the team relies on: telling quickly whether you are pressing against model limits or simply not giving the model the right information, then building toward where capability is heading. These are the team's own accounts of their history.
- tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
- Context and context window — Context is the information available for a generation; the context window is the capacity available to hold it.
- agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
- Model Context Protocol (MCP) — A published agreement for an AI application to discover what a separate program offers and ask it to run one of those offerings.
videoThe AI Frontier: from open weights to open research — Eiso Kant, Poolside AILatent Space
videoThe Agent-Native Cloud: 3M Users, 100K Signups/Wk, Data Centers, & Death PRs — Jake Cooper, RailwayLatent Space
videoAnthropic’s Felix Rieseberg on AI Coworkers, Local-First Agents, and the Future of Knowledge WorkLatent Space
Checking sign-in…
Loading comments…



