Vibeleaderboard
← Course overviewModule 01 · Understand AI · Beta
Lesson 1.3

Module contents

Understand AI

Lesson 3 of 6

What happens when you ask a question?

The model brings learned patterns from training. The application supplies information for this request. The reply depends on both, and this lesson traces how it is built.

Numbered links identify sources. Underlined terms offer a definition without leaving the lesson.

The request becomes information the model can process

Lesson two separated training a model from using it. Using a trained model to produce an output is called . You will meet that word in model descriptions and in discussions of cost, because each request takes computing time.

Your typed request is often called a . It may not be the only thing the model receives. The application can add instructions of its own, some or all of the earlier conversation, and material you attach, such as the garden note. Everything supplied for the current generation is the model’s .

Text reaches the model as : pieces that the model represents numerically, which is the numeric representation from lesson two. A token can be a whole word, part of a word or a punctuation mark, and different models divide the same sentence differently. Tokens are therefore not interchangeable with words, and a word count is only a rough guide to a token count. 5

A response is built one token at a time

The generation process does not require retrieving a finished paragraph from a library. Given its learned values and the current context, it estimates how likely each possible next token is. Software then selects one token using those estimates. The selected token is added to the context, and the process repeats: estimate, select, add. The reply grows one piece at a time until the software stops. 5

The assistant training from lesson two helps explain why a continuation takes the form of a reply to a request. It encourages that behavior alongside what the model learned from broad text. The model is still generating tokens by the same mechanism, and it can still fail to follow the instruction. 8

Return to the authored garden example. The context might be the instruction “Write a friendly reply using this note,” followed by the Saturday details. As the reply develops, the words already generated become part of what the model uses next. After “Bring,” the note makes the word “gloves” relevant to the continuation. Depending on the model, that word might require more than one token. The rain condition is also available for the reply to use, although having it in context does not guarantee it will appear. We are illustrating what information is available, not reporting measured token probabilities.

This describes the sequence of generation, not every internal calculation. Learning to continue text well appears to involve richer representations than counting which words follow which. Anthropic’s research on tracing model behavior examines some of those internal mechanisms; the field is still working out the rest. 7

Generating a text response

  1. 1 →ContextInstructions and supplied information
  2. 2 →Next pieceEstimate and select a token
  3. 03ContinueInclude the new token and generate again
This shows the sequence of generation, not every calculation inside the model.

The context has a limit

A model can only take in a fixed amount of context at once. That capacity is its context window, usually described as a number of tokens. When a conversation grows past what fits, the application must leave things out or shorten them, and the application decides how. The chat screen may still show every earlier message, but the screen is drawn by the application; it does not show which messages were actually supplied for this request. An assistant that seems to forget an instruction from early in a long conversation may simply not have received it this time. 15

The same input can produce different wording

Repeating a request can produce different wording even when nothing about the context changed. The likelihood estimates often leave several tokens as reasonable next choices. Selection software may always take the most likely one, or it may use randomness to pick among the plausible ones. A different selection at one step sends the reply down a different continuation, with no change to the learned values. Variation is possible, not guaranteed. 5

Here is an authored illustration. Two possible continuations after “Come along on Saturday at 10” are “a.m. by the north gate” and “a.m., and bring gloves.” These are pieces of a reply, not a claim that each phrase is one token or that we measured its probability. A sequence of different token choices can lead to either wording.

If you retry and the second reply includes a rain condition that the first omitted, the second reply handled that detail better. If you also reworded the request, your change may have helped. But one comparison cannot separate that influence from variation or other changes in the system. Lesson four returns to how to judge such evidence.

A correction changes the context, not the learned values

Suppose you add to the conversation: “The meeting has moved to 11 a.m.” This is a hypothetical change; the original note says 10 a.m. The next reply can say 11 a.m. because the context now contains that sentence. Nothing about the model’s learned values needs to have changed, and nothing in this observation says they did.

This distinction matters when someone says a chatbot “learned” their preference. Three different things could be going on. The preference may still be sitting in the current conversation’s context. The application may have saved it and be supplying it again in later conversations; OpenAI’s memory feature is one product example of that. Or, entirely separately, the provider may later use stored conversations when training a future model version, which depends on the service and its terms. A reply that reflects your correction shows that the information was used in that reply. It does not tell you whether the application saved it or whether a provider will use it in later training. 13 6

Put together, a better second answer has at least three candidate explanations: the context changed, the selection varied, or the application changed. Later in the course this distinction lets you read a practitioner’s tip or a model comparison without crediting one cause too quickly.

Carry this forward

  • Inference uses a trained model with the context supplied for this request; the learned values stay fixed.
  • Tokens are the pieces text is represented in. The context window bounds how many fit, and the application decides what is included.
  • A reply is built by repeatedly estimating likely next tokens and selecting one, so repeated requests can vary.
  • A changed answer can come from changed context, a different selection or a changed application; it is not evidence of retraining.

Lesson check

In one conversation you supply the note with 10 a.m.; in a separate conversation you supply a version saying 11 a.m. Each reply gives the time it was supplied. What does this observation tell you?

Choose an answer for each question.

This is a practice check. Answers reset when you leave this page. Results are not saved and do not place you in another module or award a qualification.

We can now trace how a reply is produced. We still need to separate a plausible reply from a supported one.

Sources · Reviewed 2026-09-11
  1. 5. Google: Language models and tokens
  2. 7. Anthropic: Tracing language-model behavior
  3. 13. OpenAI: Memory explained
  4. 6. OpenAI: How its models are developed
  5. 8. Anthropic: Model reports
  6. 15. Anthropic: what enters the context window

Sources support nearby explanations. Product details can change. Scenarios and model results are authored illustrations unless a lesson explicitly identifies a recorded execution. Module 7 identifies its offline code checks.