Intel / article

GPT-6 Astra, Looped Transformers, and Hidden Reasoning

Source: magazine.sebastianraschka.com
Author: Sebastian Raschka, PhD
Date: Published Sep 9, 2026

From the Daily Brief

AI researcher Sebastian Raschka examines two ideas that have become tangled in discussion of GPT-6 Astra: recurrent-depth transformer architecture and hidden reasoning traces. A looped transformer reuses one or more blocks, passing an internal representation through them repeatedly. That adds computation within the network without requiring a separate, visible reasoning token for every step. A chain of thought is different: it is the sequence of generated tokens a reasoning model uses as a scratchpad before producing its answer. Raschka notes that Astra can use fewer output tokens than GPT-5.6 Sol at a fixed accuracy, but argues that this does not by itself show that more reasoning has moved into an inaccessible internal process. A stronger model may simply need less backtracking. His conclusion is appropriately limited because OpenAI has not published enough architecture detail to establish why Astra behaves as it does. For developers evaluating the model, measured task performance and token use are observable; claims about hidden internal reasoning remain hypotheses unless the model maker supplies stronger evidence.

Read the 2026-09-09 Brief →

Context

A looped performs extra internal computation by passing intermediate representations through the same processing layers repeatedly. It can therefore add processing steps without storing a separate set of weights for every step. The Universal Transformer paper describes this recurrence over successive revisions of the model's internal representations.

Sebastian Raschka uses that distinction to examine claims about hidden reasoning in OpenAI's GPT-6 Astra. The article notes that reports of Astra's looping lack official confirmation, though the author thinks it likely, and argues that shorter written reasoning does not establish looping as its cause: a more capable model may simply need fewer written steps. Internal computation and a model's written explanation are different things; knowing that one changed does not identify how the other was produced.

Terms in this piece · Glossary

transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.

Key quotes

“I want to prevent a race into unmonitorability kicked off by confused reporting.”
Jakub Pachocki

“The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.”
Jakub Pachocki

“Looped transformers simply give better modeling performance at a fixed compute budget.”

Read the source magazine.sebastianraschka.com