Intel / article

Why We Think

Source: lilianweng.github.io
Author: Lilian Weng
Date: Published May 1, 2025

Context

Giving an AI model more time to answer requires a method for using the extra computation. Lilian Weng's 2025 survey describes two approaches: generate several candidate solutions and select among them, or revise one solution through successive attempts.

These approaches face different constraints. Multiple candidates help only if the model can produce a useful solution and the selection process can recognize it. Revision can respond to feedback, but ungrounded self-correction can also replace a correct answer with an incorrect one. Extra computation becomes useful through the search and checking process surrounding generation; a longer response alone does not establish better reasoning.

Terms in this piece · Glossary

chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.

Key quotes

“In Transformer models, the amount of computation (flops) that the model does for each generated token is roughly 2 times the number of parameters. For sparse models like mixture of experts (MoE), only a fraction of the parameters are used in each forward pass, so computation = 2 * parameters / sparsity, where sparsity is the fraction of experts active.”
Lilian Weng

“Snell et al. (2024 ) showed that easier questions benefit from purely sequential test-time compute, whereas harder questions often perform best with an optimal ratio of sequential to parallel compute.”
Lilian Weng

“Interestingly the DeepSeek team showed that with pure RL, no SFT stage, it is still possible to learn advanced reasoning capabilities like reflection and backtracking (“Aha moment”). The model naturally learns to spend more thinking tokens during the RL training process to solve reasoning tasks.”
Lilian Weng

“They failed to use process reward model (PRM) as it is hard to define per-step rubrics or determine whether an intermediate step is correct, meanwhile making the training more vulnerable to reward hacking.”
Lilian Weng

“Chain of thought provides an especially convenient form of interpretability, as it makes the model’s internal process visible in natural language. This interpretability, however, rests on the assumption that the model truthfully describes its internal thought processes.”
Lilian Weng

Read the source lilianweng.github.io