Vibeleaderboard
← All Intel
Intel / video

Notion’s Sarah Sachs & Simon Last on Custom Agents, Evals, and the Future of Work

Source
youtube.com
Author
Latent Space
Date

Context

Notion's Simon Last and Sarah Sachs recount in this Latent Space interview that the company tried to build an agent that could use all of Notion's tools as early as late 2022, when it first had access to GPT-4, and kept failing. Last says there was no common idea of yet, the models were not capable enough and windows were too short. The team designed its own tool-calling framework and partnered with Anthropic and OpenAI at different times to fine-tune models to use it over multiple turns.

Last says the unlock was a Claude Sonnet release recalled as 3.6 or 3.7, after which Notion built the agent it shipped. Custom agents, which run in the background, took longer because the team wanted far higher reliability and a permissions design for cases where a shared agent's audience and its document access differ. Sachs describes a the team relies on: telling quickly whether you are pressing against model limits or simply not giving the model the right information, then building toward where capability is heading. These are the team's own accounts of their history.

Terms in this piece · Glossary
  • tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
  • Context and context window — Context is the information available for a generation; the context window is the capacity available to hold it.
  • agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
  • Model Context Protocol (MCP) — A published agreement for an AI application to discover what a separate program offers and ask it to run one of those offerings.
Read the source www.youtube.com
More from Latent Space
Recommended reads
Comments

Checking sign-in…

Loading comments…