Why AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax
Source
AI Engineer
Author
AI Engineer
Date
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
Why it matters
Explains the sparse-attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition → design giving MiniMax M3 a functional million-tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition →context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → alongside coding and multimodalA model that works with more than text — reading images, audio, or video, and sometimes generating them too.Full definition → capability, directly relevant to agents that must reason over long tool-call histories.
videoYour company brain will leak secrets: how we stopped it for big banks — Tanmai Gopal, PromptQL
videoTethered: Our Agents Are Us — Shu Fang, Two Sigma
videoAgents' next frontier: agent-to-agent and network effects — Jean-Denis Greze, Town
videoEveryone Gets A Software Company — Benjamin Guo, Zo Computer
