← All IntelClip / EducationBase versus instruct, and why point releases jump on benchmarks
From Training an LLM from Scratch, Locally — Angelos Perivolaropoulos, ElevenLabs · ≈1:05:21
A candid read on fast industry gains — the same pretrained base plus better post-training data aimed at problems that resemble the current benchmarks.
What’s in it
- A candid read on fast industry gains — the same pretrained base plus better post-training data aimed at problems that resemble the current benchmarks.
Clip transcript
Uh the microphone is not working. Is it? Yeah, perfect. Uh so, reasoning and non-reasoning models share the same base often, and then one is post-trained in a different way and add this uh you know, in a way different post-training. Yes. So, a a lot of the labs, like let's say the current model that was released like 1 3, usually they release like a base version, an instruct version, and the usually the base version doesn't have a chain of thought reasoning. Uh it's usually like the same model that they first pre-train to be like quite good, maybe do some fine-tuning as well. And then the next step is doing post-training to teach it this kind of like this this uh this knowledge. Uh so, that's why like you see a lot of a lot of uh big improvements that happen very fast in the industry, like Gemini 3 to 3.1, it's essentially giving it like better reasoning data and like like better fine-tuning post-training data for like specific problems. It's a little bit like benchmarking, like usually this data is like very similar to what the benchmarks, the current good benchmarks are. But yeah, essentially just taking a base model, using this new data to improve it to the next level.
Comments
Sign in to comment.
Loading comments…