Vibeleaderboard
← All Intel
Intel / video

“A Staff Engineer Collaborator” | Notion’s First Look at GPT-6 Astra

Source
youtube.com
Author
OpenAI
Date

Context

An AI application can keep answering correctly while quietly losing the savings from , which reuses processing of repeated input. In OpenAI's account of a Notion cache audit, GPT-6 Astra connected a pattern the team at Notion had overlooked: older messages were being regenerated using what the knew, subtly changing the prompt history and breaking cache reuse. The description says everything could still appear to work while becoming slower and more expensive at scale.

The mechanism matches the prompt-caching documentation: cache reuse requires the entire rendered prefix to match, so rewriting earlier turns invalidates the reusable portion, and the guidance is to append new messages rather than rewrite earlier ones. The audit illustrates why message-history construction deserves even when the application's answers still look right. No measured savings are reported.

Terms in this piece · Glossary
  • prompt caching — Reusing the model's processed form of a repeated prompt prefix so subsequent calls skip re-reading it, cutting cost and latency substantially.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
Read the source www.youtube.com
More from OpenAI
Recommended reads
Comments

Checking sign-in…

Loading comments…