#459 – DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters
Source
Lex Fridman
Author
Lex Fridman
Date
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
A deep technical breakdown of how DeepSeek achieved frontier-level models at dramatically lower training cost, plus the GPU supply/export-control realities shaping who can build large clusters — useful context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → for anyone reasoning about model economics and compute strategy.
Key quotes
“OpenAI had such little compute and they devoted all of their compute for many months, all of it, 100% for many months to GPT-4 with a brand-new architecture”
Dylan Patel
“The big winners throughout human history are the ones who are willing to do YOLO at some point.”
Lex Fridman
“And language models crash the cost of very intelligent sounding language.”
Dylan Patel
“DeepSeek is doing fantastic work for disseminating understanding of AI. Their papers are extremely detailed in what they do and for other teams around the world, they're very actionable in terms of improving your own training techniques.”
Nathan Lambert
“They freaking have F-35s and we don't let them buy GPUs.”
podcast#494 – Jensen Huang: NVIDIA – The $4 Trillion Company & the AI Revolution
podcast#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
podcast#475 – Demis Hassabis: Future of AI, Simulating Reality, Physics and Video Games
podcast#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI