Why AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax

In a conversation with AI Engineer, MiniMax's RL lead Thomas Wolf and Hugging Face co-founder Olive Song walk through the sparse-attention architecture behind MiniMax M3's million-token context window, explaining how the model was trained multimodal from the first step rather than retrofitted afterward. They describe the practical effect: a context window long enough to hold entire codebases or tool-call histories without truncation, which matters directly for agents that need to reason over long, branching action logs rather than short, isolated prompts. The two also discuss how M3 is already being used to accelerate work on its own successor, M3.1, an early example of a model materially speeding up its own development cycle. For engineers evaluating long-context options beyond the usual frontier labs, the interview is a useful look at the architectural tradeoffs MiniMax made to get there and what it unlocks for agent designs that lean on extended memory instead of retrieval.