Vibeleaderboard
← All Intel
Intel / article

Perplexity and Intelligence

Source
openai.com
Author
acossta
Date

Context

OpenAI's chief scientist, Jakub Pachocki, proposes that AI labs agree on one number to track how capable their models are getting, and that they commit to development limits measured partly by it. The number is perplexity: how well a model predicts the next token of text it has never seen, where lower means better prediction. OpenAI tracks its logarithm, also called cross-entropy loss, on a held-out dataset drawn from its own internal codebase. Pachocki's case is that , teaching a model to predict a huge body of data, is where general ability comes from, and that later steps such as and reinforcement learning mostly draw out abilities the pretrained model already has.

To show that the number tracks something real, he publishes OpenAI's own figures, which he says took some consideration because it tells competitors where OpenAI stands. GPT-3.5, the basis of the original ChatGPT, sat at a log-perplexity of about 0.99 on the current dataset. GPT-4, at about 0.91, could use step-by-step reasoning productively, which led to today's reasoning models. Below 0.8, he writes, OpenAI saw transformative capabilities in cybersecurity and mathematics, and after moving to a model at about 0.76, open math problems OpenAI had been posing to its models started coming back with what he calls groundbreaking results. He stresses that nobody knew in advance those abilities would appear at those thresholds. Extrapolating, he expects further transformative capabilities in the 0.68 to 0.7 range, without knowing what they will be, along with weaker auditing and models more aware that they are being evaluated or trained.

The proposal follows from that worry. He wants frontier developers to set shared norms for evaluating and red-teaming such models, plus common limits, based on perplexity and other measurements, that labs would not cross until their ability to understand and monitor model behavior reaches a concrete milestone, such as a successor to chain-of-thought monitors. He also raises the possibility that AI doing pretraining research could bring an avalanche of advances as soon as 2027, though he attributes most of OpenAI's steep 2026 progress to long-term research and compute investments, and only in a smaller part to automation.

He lists the metric's weaknesses himself. Labs use different evaluation data and training mixes, so numbers do not yet compare across companies. Perplexity measures no task anyone cares about directly. It misses gains outside pretraining; chain-of-thought reasoning sharply raises capability without lowering perplexity. And no one knows how low it can go, so progress will look slower as models near that floor even when each small gain means a larger jump in ability. The figures above are OpenAI's own, on its private code dataset, and cannot be checked from outside.

Terms in this piece · Glossary
  • pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
  • fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
  • chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
  • alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
Recommended reads
Comments

Checking sign-in…

Loading comments…