RLVR scores only the final answer, 0 or 1, using a symbolic checker for math and a compiler or unit tests for code. DeepSeek tried feeding the reasoning trace itself into the training signal and found it did not help, so R1 threw it away.
DeepSeek-R1-Zero showed reinforcement learning applied straight to a base model, with no supervised fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → first, is enough for the model to start writing step-by-step traces, backtracking and catching its own errors. It scored below full R1, but it worked.
The <think> tags are cosmetic. They mark where the trace starts and stops so the pipeline or UI can hide it; a model trained without them benchmarks about the same. Models emit them because RLVR adds a format reward alongside the accuracy reward.
Reasoning training is already a form of inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → scaling, since a reasoning modelA model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.Full definition → simply spends more tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → per answer. Self-consistency stacks on top: sample several answers and take the majority vote, which works on ordinary LLMs too.
Terms in this piece · Glossary
reasoning model — A model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Why it matters
If you tune reasoning effort in models like GPT-5, Qwen3, or gpt-oss, this deep-dive explains how those low/medium/high modes are actually trained and implemented under the hood via RLVR and think tokens—turning a black-box toggle into something you can reason about.
Key quotes
“These <think> and </think> tags are cosmetic with respect to reasoning ability. They do not make the model reason, and they are not required to achieve good reasoning performance.”
“So yes, reasoning models are here to stay. They have become a standard part of modern model releases.”
“a smaller model at a higher reasoning effort can sometimes reach a similar score as a larger model at a lower reasoning effort”
“The holy grail is of course automatic effort selection.”
