“A Staff Engineer Collaborator” | Notion’s First Look at GPT-6 Astra
- Source
- youtube.com
- Author
- OpenAI
- Date
Context
An AI application can keep answering correctly while quietly losing the savings from , which reuses processing of repeated input. In OpenAI's account of a Notion cache audit, GPT-6 Astra connected a pattern the team at Notion had overlooked: older messages were being regenerated using what the knew, subtly changing the prompt history and breaking cache reuse. The description says everything could still appear to work while becoming slower and more expensive at scale.
The mechanism matches the prompt-caching documentation: cache reuse requires the entire rendered prefix to match, so rewriting earlier turns invalidates the reusable portion, and the guidance is to append new messages rather than rewrite earlier ones. The audit illustrates why message-history construction deserves even when the application's answers still look right. No measured savings are reported.
- prompt caching — Reusing the model's processed form of a repeated prompt prefix so subsequent calls skip re-reading it, cutting cost and latency substantially.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
Checking sign-in…
Loading comments…






