Controlling Reasoning Effort in LLMs
- Source
- magazine.sebastianraschka.com
- Author
- Sebastian Raschka, PhD
- Date
Context
A reasoning-effort setting can change how an existing model responds without selecting a different model. For OpenAI's gpt-oss, the official documentation specifies low, medium and high levels controlled through the .
Sebastian Raschka's analysis explains how training could make such instructions meaningful: examples can associate effort labels with different response lengths, or training rewards can apply different penalties for lengthy answers. Raschka presents these as possible implementations, not a disclosed proprietary training recipe. The setting asks a model to allocate effort differently; simply adding an effort label to an arbitrary model does not establish the same control.
- RLVR scores only the final answer, 0 or 1, using a symbolic checker for math and a compiler or unit tests for code. DeepSeek tried feeding the reasoning trace itself into the training signal and found it did not help, so R1 threw it away.
- DeepSeek-R1-Zero showed reinforcement learning applied straight to a base model, with no supervised first, is enough for the model to start writing step-by-step traces, backtracking and catching its own errors. It scored below full R1, but it worked.
- The <think> tags are cosmetic. They mark where the trace starts and stops so the pipeline or UI can hide it; a model trained without them benchmarks about the same. Models emit them because RLVR adds a format reward alongside the accuracy reward.
- Reasoning training is already a form of scaling, since a simply spends more tokens per answer. Self-consistency stacks on top: sample several answers and take the majority vote, which works on ordinary LLMs too.
- system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
- reasoning model — A model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.
- fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
“These <think> and </think> tags are cosmetic with respect to reasoning ability. They do not make the model reason, and they are not required to achieve good reasoning performance.”
“So yes, reasoning models are here to stay. They have become a standard part of modern model releases.”
“a smaller model at a higher reasoning effort can sometimes reach a similar score as a larger model at a lower reasoning effort”
“The holy grail is of course automatic effort selection.”
articleDiverse reasoning traces teach LLMs to make better decisionswww.amazon.science
articleCategories of Inference-Time Scaling for Improved LLM ReasoningSebastian Raschka, PhD
articleThe Thinking Leakage in Reinforcement Learning for LLMsWenxuan Zhu, Shuming Liu, Bang An, Aznaur Aliev, Bing Li, Yibo Yang, Bernard Ghanem
Checking sign-in…
Loading comments…