Vibeleaderboard
← All Intel
Intel / article

An Operationalization of Opaque Serial Depth

Source
arxiv.org
Author
Nathan Sheffield, Alek Westover, Lukas Finnveden, Alexa Pan, Julian Stastny, Ryan Greenblatt
Date

Context

A group of safety researchers proposes a number that AI developers could publish before training a model, to show how much reasoning its design lets it do without that reasoning ever appearing as readable text. Today's reasoning models work through problems in a chain of thought, text written out step by step that people and monitoring software can read, and much of current AI oversight relies on reading it. The worry is that new architectures could let a model carry long trains of thought in internal numbers instead. In An Operationalization of Opaque Serial Depth, Nathan Sheffield, Ryan Greenblatt and four co-authors, writing from Redwood Research, build on a measure that researchers at Google DeepMind recently defined: opaque serial depth, the longest chain of computation steps that never passes through an interpretable bottleneck, a point where information has to be squeezed into something a person can read. Their contribution is a precise rule for what counts as such a bottleneck, and they call the resulting number NLS depth.

Under that rule, generated tokens count as a readable checkpoint only if they stay tied to what taught them to mean. The model must have been trained first to produce human-understandable media (the authors include code, JSON and images), it must emit only tokens whose meanings came from that training, and it must not have been altered to use those tokens as a code for some other kind of data; gradients from later steps must also not be passed back through them. Prompting, and reinforcement learning do not break the rule, even when they make the chain of thought hard to read in practice. That is deliberate, because it lets anyone work out NLS depth from the architecture, training recipe and deployment setup before training starts. Depth is counted in an idealized circuit of operations on pairs of 32-bit numbers, so it measures how many steps must happen one after another, not total computation; the paper's worked example puts the small open-weight model Gemma3-1B at no more than 1,954. For a standard transformer reasoning in text, the hidden stretch is a single forward pass, so a long reasoning session does not add to the depth. Designs that feed hidden states back into the model multiply it instead: 1,000 latent reasoning steps in COCONUT, a continuous-thought method from Meta researchers, would give 1,000 times the depth of one pass.

Measured this way, open-weight models have barely deepened. NLS depth tracks layer count closely, no open-source model exceeds 17,000, and the authors say they would be surprised if any closed model were more than twice that. Depth has grown roughly with the quarter power of a model's active parameters, and compute-optimal training grows active parameters with about the square root of training compute, so depth would rise with roughly the eighth root of compute. If training compute keeps rising about fivefold a year and that relationship holds, the authors project about 1.22 times more depth per year, a doubling every three to four years. Latent-reasoning designs such as full-bandwidth transformers, which carry hidden state across the whole , could jump far past that. The authors tie the question to OpenAI's GPT-6 Astra, whose system card reports substantially lower chain-of-thought monitorability than OpenAI's earlier models, while OpenAI's chief scientist has said its computation graph is within a factor of two of GPT-4's depth. The paper calls the cause unclear and says only that public evidence is consistent with higher depth being a significant contributing factor; Sebastian Raschka's explainer on Astra and looped transformers covers the architecture rumors behind that question.

NLS depth is a proxy, and the paper documents where it misfires. DeepSeek-V4-Pro scores high only because each of its 61 layers runs two repeated normalization routines that add 12,200 to its depth while handling 16 numbers and no learned parameters; the authors call that a flaw in their definition. Kimi-K3's higher depth, which comes from its DeltaNet-style attention, they treat as a real but moderate concern. The measure also ignores whether a chain of thought stays legible, so a model whose reasoning text drifts into gibberish keeps a low score, and it does not register deeply recurrent models switching from talking to each other in text to passing hidden states, for which the authors suggest a separate count of total hidden computation. They present NLS depth as a complement to direct tests of monitorability, not a replacement, and ask developers to track and report it.

Key takeaways · AI-distilled
  • The authors define opaque serial depth as roughly the longest computation path that skips every interpretable bottleneck, and count as bottlenecks only nodes initialized from a pretrained prior to emit human-readable output whose output space and meaning stay unchanged.
  • Their measure, NLS depth, can be judged before training from architecture, training recipe and deployment configuration alone, and by construction ordinary post-training such as prompting, distillation and RL does not change it.
  • Across open-weight models, the abstract reports that NLS depth tracks layer count closely, with some exceptions, and has grown slowly in recent years.
  • The authors expect standard transformer scaling to raise NLS depth only slowly, while latent-reasoning designs such as full-bandwidth transformers could raise it a lot. They bound Gemma3-1B's depth as a worked example.
  • They argue developers should track and report NLS depth alongside direct measurements of chain-of-thought monitorability.
Terms in this piece · Glossary
  • pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
  • distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
  • chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Recommended reads
Comments

Checking sign-in…

Loading comments…