#459 – DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters

Downloading an AI model and reproducing its training are different levels of access. Discussing DeepSeek, AI researcher Nathan Lambert explains that open weights provide the learned numerical parameters that make the model work. They do not necessarily include the data or software used to train it. The interview lays out that distinction.

Lambert argues that missing training data and code make replication harder even when weights and a detailed technical report are available. For someone evaluating an open release, the useful questions therefore extend beyond download availability: what can be run, what can be studied or rebuilt, and what uses does the license permit?

“OpenAI had such little compute and they devoted all of their compute for many months, all of it, 100% for many months to GPT-4 with a brand-new architecture”

Dylan Patel

“The big winners throughout human history are the ones who are willing to do YOLO at some point.”

Lex Fridman

“And language models crash the cost of very intelligent sounding language.”

Dylan Patel

“DeepSeek is doing fantastic work for disseminating understanding of AI. Their papers are extremely detailed in what they do and for other teams around the world, they're very actionable in terms of improving your own training techniques.”

Nathan Lambert

“They freaking have F-35s and we don't let them buy GPUs.”

Dylan Patel