Pretraining is where a model absorbs most of what it knows — months of computation over trillions of tokens, costing up to hundreds of millions of dollars. The output is a "base model": knowledgeable but raw, good at continuing text rather than following instructions.

Everything after — instruction tuning, RLHF, fine-tuning — is comparatively small sculpting on top of that mass. When people say a capability "comes from pretraining," they mean no amount of later polish adds it if it isn't already latent in the base.