TTT-E2E Lets LLMs Keep Training on Context During Deployment
- Source
- x.com
- Date
Our latest research, TTT-E2E, marks a new era for LLM memory. Now, models can continue training during deployment, using context as training data to update their weights and learn from massive amounts of experience. With @NVIDIAAI and @AsteraInstitute https://t.co/wEBjbDy8RT
Context
A standard either caches every past token for , which gets expensive as grows, or uses a limited window and loses information beyond it. In its announcement, Stanford SAIL describes TTT-E2E as instead compressing context into the model's own weights on the fly, training on the incoming text via next-token prediction rather than only reading it as a static prompt.
The paper reports that on a 3-billion-parameter model trained with 164 billion tokens, this approach scales with context length similarly to full attention, unlike comparison models such as Mamba 2 or Gated DeltaNet, while keeping roughly constant latency regardless of context length, reported as 2.7 times faster than full attention at 128K tokens of context. These are the authors' own reported benchmark results at that specific model scale, not an independent replication.
- transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
- attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Checking sign-in…
Loading comments…