Building Reasoning Models from the Ground Up: A Video Course
Source
magazine.sebastianraschka.com
Author
Sebastian Raschka, PhD
Date
Why it matters
Walks through building and improving a reasoning model from a pre-trained LLM in PyTorch, covering inference-time scaling, RL, and distillation. Useful for engineers who want to understand how the reasoning modes they build on actually work.
Key takeaways · AI-distilled
Raschka frames a reasoning modelA model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.Full definition → as a conventional LLM given extra post-training through reinforcement learning with verifiable rewards, optionally plus inference scaling, while the agent harness around it supplies context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → management and tool access.
The course starts from a pre-trained Qwen3 0.6B model instead of training one, so its eight videos (about 11 hours) spend their time on evaluation, test-time computeSpending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.Full definition →, reinforcement learning and distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition →.
One math-answer verifier does double duty: it extracts, normalizes and checks answers for mathematical equivalence to score the model on MATH-500, then supplies the reward signal for RLVR training.
Inference-time scaling is built from temperature, multinomial and top-p sampling plus self-consistency majority voting, then log-probability scoring and a self-refinement loop, with MATH-500 accuracy weighed against the extra inference compute.
The RL lectures implement a simplified GRPO, then add clipped policy ratios, a KL term and format rewards conditioned on correctness; the last lecture distills from DeepSeek R1 answers as an alternative to RL.
Terms in this piece · Glossary
reasoning model — A model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
