Trillion Token Context. No, Really — Anima Anandkumar & Benedikt Jenik, Accelerated Understanding

Introduces a new lab's push toward extremely long context for science models, relevant to anyone tracking context-length limits and foundation models beyond language.

• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
• context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

• videoInside xAI: Building Grok Imagine in 3 Months, Videogen vs World Models, and Video Agents— Ethan He
• videoBest of 2024 in Agents (from #1 on SWE-Bench Full, Prof. Graham Neubig of OpenHands/AllHands)
• videoThe Unreasonable Effectiveness of Reasoning Distillation: using DeepSeek R1 to beat OpenAI o1
• videoAI Agents Need Computers: 74% MoM Growth, 850K/Day Runs, & New Agent Cloud — Ivan Burazin, Daytona

• video🔬 Why Transformers Hit a Wall the Moment Physics Shows Up — Anima Anandkumar, CaltechLatent Space
• videoInterpretability: Understanding how AI models thinkAnthropic
• article🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of ComputingBrandon Anderson

Checking sign-in…

Loading comments…