GPT-6 Astra, Looped Transformers, and Hidden Reasoning
Source
Sebastian Raschka, PhD
Author
Sebastian Raschka, PhD
Date
Terms in this piece · Glossary
transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
Why it matters
Explains looped/recurrent-depth transformerThe neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.Full definition → architecture and whether it accounts for GPT-6 Astra's hidden reasoning trace, giving engineers a concrete framework for interpreting future reasoning-hidden model releases.
Key quotes
“I want to prevent a race into unmonitorability kicked off by confused reporting.”
Jakub Pachocki
“The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.”
Jakub Pachocki
“Looped transformers simply give better modeling performance at a fixed compute budget.”
