Vibeleaderboard
← All Intel
Intel / video

Stanford CS25: V5 I On the Biology of a Large Language Model, Josh Batson of Anthropic

Source
youtube.com
Author
Stanford Online
Date

Context

Joshua Batson, who leads the circuits effort on Anthropic's interpretability team, argues in a Stanford CS25 lecture that studying language models resembles biology: they are grown by training rather than designed, so researchers study what the finished system does inside.

He says single neurons are not enough, because the text that makes one neuron fire is a mix of code, Chinese and math. He instead sets out three claims about models that an outside view misses: they build and combine abstract internal representations, they run many computations in parallel rather than one step at a time, and they plan many words ahead even though they write one word at a time. This covers the opening of the talk, where these are stated as claims; the supporting experiments are not covered here. For Anthropic's own account, see this interpretability interview.

Terms in this piece · Glossary
  • LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Read the source www.youtube.com
More from Stanford Online
Recommended reads
Comments

Checking sign-in…

Loading comments…