DeepSeek reset cost intuitions: roughly $5M of compute for the V3 base run and another $294K to train R1 on top. The caveat is that this covers the final run only, excluding salaries, failed experiments, and hyperparameter search.
RLVR works because a deterministic program can check correctness, so post-training stops being bottlenecked by hand-written responses or human preference labels and starts scaling with compute the way pretrainingThe first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.Full definition → does.
The dominant technique rotates roughly yearly: RLHFReinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.Full definition → with PPO in 2022, LoRAA cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.Full definition → fine-tuning in 2023, mid-training (synthetic data, data mixes, long-context stages) in 2024, RLVR with GRPO in 2025. Each stays in use, it just stops being the frontier.
Process reward models, which grade each reasoning step rather than only the answer, were tried and dropped in DeepSeek R1 because the gain didn't justify the compute overhead at RL scale. DeepSeekMath-V2 suggests scored explanations are returning.
Continual learning is blocked by catastrophic forgetting: continued pretraining shows a model absorbing new knowledge loses some old knowledge, which is why the topic stayed hot all year without a breakthrough.
Terms in this piece · Glossary
test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
RLHF — Reinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.
LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
Why it matters
It distills a chaotic year of LLM research — DeepSeek's cost disruption, the rise of RLVR/GRPO for reasoning, and test-time computeSpending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.Full definition → — into one technically authoritative synthesis, giving practitioners a grounded map of where models are heading in 2026/2027.
Key quotes
“If I had to pick a word or trend that describes LLM development this year, it would be ",”
