Vibeleaderboard
← Course overviewModule 01 · Understand AI · Beta
Lesson 1.4

Module contents

Understand AI

Lesson 4 of 6

Why can a convincing answer be wrong?

A reply can read well while claiming something the available information does not support. Fluency and evidence answer different questions.

Numbered links identify sources. Underlined terms offer a definition without leaving the lesson.

A plausible continuation is not a checked fact

Lesson three showed a reply built by choosing likely next tokens from learned patterns and supplied context. That process can produce useful text, but generating a statement does not by itself show that it was checked against suitable evidence. A model may lack a fact, misread the context or add a detail that fits the usual shape of a reply. Generated content that is false or unsupported is often called a .

Training explains part of the risk. Predicting a plausible continuation rewards producing something, including when the model has no reliable answer. Assistant training may encourage the model to express uncertainty or decline, but mistakes can remain. Anthropic’s research describes cases where a model produces a plausible answer without sufficient knowledge and examines some of the mechanisms behind that behavior. Those findings cover the models and tasks studied, not every error a model can make. 7

Check the claim against what was supplied

Here is the original fictional note again:

Saturday: meet at 10 a.m. by the north gate. Bring gloves. The session is cancelled if it rains.

Now consider this authored reply:

Join us at 10 a.m. on Saturday by the north gate. Gloves and lunch will be provided.

Read the reply against the note, claim by claim. The first sentence matches the note. “Gloves will be provided” turns an instruction to bring gloves into a promise; the note says nothing about spare gloves, so the promise is unsupported. It is not disproven, because the note could simply be silent about spares, but nobody should tell the neighbor to leave their gloves at home on this evidence. Lunch appears nowhere in the note. The rain condition is missing, which is an omission rather than a false claim, and it matters: a neighbor who turns up in the rain finds no session.

A useful check locates each problem instead of calling the whole reply “bad.” A good correction keeps the meeting details, tells the neighbor to bring gloves, includes the cancellation condition and drops the lunch promise unless another reliable source supports it. This check needed nothing except the note itself.

Check a reply claim by claim

  1. 0110 a.m.Supported by the garden note
  2. 02Gloves providedNot promised by the garden note
  3. 03Lunch includedNot established by the note
An unsupported addition is not the same as a fact that has been disproven.

Different claims need different checks

Some claims cannot be checked against the note at all, and those need the right kind of evidence. Whether the session has since been cancelled is a question about now. The old note cannot answer it; a current notice from the organizer can. A confident restatement of the original plan is not evidence either way.

An application may bring in outside information. ChatGPT, for example, can search the web and include links to what it found. That is a capability of the software around the model. It does not mean every answer was looked up, and a link is only useful if the page behind it actually says what the nearby sentence claims and is recent enough for the question. Sources can be wrong too; a citation moves the question to whether that source is reliable rather than settling it. 14

Arithmetic needs a different check again. Suppose you ask how long three beds would take using the earlier rule of 20 minutes per bed. Check the inputs and the operation: three times twenty is sixty minutes. Anthropic has studied models performing some arithmetic internally, but a correct result on one task does not make every answer reliable. Whether an answer came from a model or a calculator, check that the right numbers and operation were used. 7 A correct multiplication also still rests on whether that simplified rule fits the job.

Asking the same system whether it is sure may produce reassurance without any new evidence. A useful recheck would return to the note, inspect a source or perform the calculation. Judge that work rather than the confidence of the answer; a second confident statement alone is not an independent check.

Make the conclusion match the evidence

The same habit applies to claims about AI tools, including the practitioner reports this course will cite. A practitioner’s account can tell you what they tried and what they saw. A stated preference tells you which tradeoff they value. Neither shows that the same approach would improve a different setup, and several people repeating one report are not several independent observations.

Remember from lesson three that repeated outputs can vary. If a technique is tried once and the next reply is better, that single retry cannot separate the technique from ordinary variation. You can still try a promising approach; the discipline is to keep the uncertainty visible while you do.

For the garden message, the practical conclusion is narrow and achievable: check that the reply preserves the supplied details, and treat anything beyond them as unsupported until you find evidence.

Carry this forward

  • A reply can hold correct details, an unsupported addition and an omission at the same time; check claim by claim.
  • Unsupported is not the same as disproven. A silent note leaves a claim open, not refuted.
  • Match the check to the claim: the supplied note, a current source or a calculation. Reassurance alone is not an independent check.
  • A single improved retry does not identify why it improved.

Lesson check

The garden note gives a meeting time, an instruction to bring gloves and a rain condition. A reply adds “Lunch is included.” What is the strongest conclusion you can draw from the note alone?

Choose an answer for each question.

This is a practice check. Answers reset when you leave this page. Results are not saved and do not place you in another module or award a qualification.

Knowing what a reply does and does not support prepares you to read the announcements made by the organizations that develop and supply models.

Sources · Reviewed 2026-09-11
  1. 7. Anthropic: Tracing language-model behavior
  2. 14. OpenAI: Search in ChatGPT

Sources support nearby explanations. Product details can change. Scenarios and model results are authored illustrations unless a lesson explicitly identifies a recorded execution. Module 7 identifies its offline code checks.