← All IntelClip / AI ToolsOrigin of 'thinking tokens' as internal reasoning
From Scaling to Long Horizons — Ross Taylor & Chengxi Taylor, General Reasoning · ≈5:51
“Galactica was really this first idea which said, "No, this is an internal working memory process. This is an internal thinking. You should be inside these tags, and you should spend the inference computer before you get to an answer, right?"”
What’s in it
- Traces the origin of 'thinking tokens' back to Meta's Galactica paper
- Explains how chain-of-thought and scratch-pad prompting differed from thinking tokens
- Shows Galactica as the first model to treat reasoning as internal working memory
Clip transcript
first like real empirical result for that. Now, perhaps more importantly, there's this idea of thinking tokens. And some of you might remember this, but it was like really quite buried within the paper. So, around this time there were like different ideas for reasoning. There was chain of thought, which was one idea. It's where you prompt for like kind of like the steps. There were scratch pads where you just like put like very like numerical kind of intermediate steps. But, Galactica was really this first idea which said, "No, this is an internal working memory process. This is an internal thinking. You should be inside these tags, and you should spend the inference computer before you get to an answer, right?" So, these are all like quite like pressing
Comments
Checking sign-in…
Loading comments…