Vibeleaderboard
← Course overviewModule 03 · From answering to acting · Beta
Lesson 3.5

Module contents

From answering to acting

Lesson 5 of 5

What do I do when it goes wrong?

The fencing trace showed a failed retrieval and a proposed action outside the task. It also showed why a successfully written table still needs a correctness check. This lesson sorts failures by where they happen in the loop and what recovery each allows, then closes the module with a transfer task in a different setting.

Numbered links identify sources. Underlined terms offer a definition without leaving the lesson.

Failures happen at different points in the loop

An agent can fail in five different places, and the evidence for each lives somewhere different.

An operation can fail or its outcome can be unknown. A file-not-found response reports that the requested read returned no file at that path. A timeout reports that a response did not arrive in time; it does not establish whether a remote server acted. The recovery decision depends on the particular result and the operation's possible effects.

An operation can succeed and not serve the request. Module 2 built its whole final lesson on this: a listing by modification date that was asked for documents started this month, a site search that could not cover publication elsewhere. 8 The evidence is in comparing the operation the record shows with the request that was made.

The model can misread a correct observation. The Hedgerow total is the example: right numbers in the record, wrong sum in the table. The evidence is in comparing the output with the observations.

The model can propose an action outside the task. The email at line 19 and the imagined typo fix are both this. The evidence is in comparing the proposal with the brief, at the prompt if there is one, and in the record afterwards if there was not.

The loop can end without the work done. A step limit is reached, the model declares completion early, or a person stops it. The evidence is in comparing the record with the completion evidence the brief named.

Naming the place matters because it decides what to do. A failed operation calls for a decision about retrying. A misread observation calls for correcting the output. An out-of-task proposal calls for a change to the rules. Treating all five as “the agent got it wrong” loses that.

Retrying, substituting and stopping

Take the failed fetch. A timeout is a plausible transient failure, the kind that may not recur, so one retry is reasonable, and the brief allowed one. Another retry might eventually succeed, but would exceed this brief's allowance and consume more resources. Substituting a different action, such as emailing the supplier, crosses the line the brief drew. Stopping and reporting is what the brief asked for, and it hands a decision that involves contacting an outside party back to the person who can make it.

Whether a retry is safe depends on what the operation does. A genuinely read-only operation usually avoids duplicating a change, but can still consume time, paid calls or a service's request allowance. Repeating an action with an , meaning a consequence outside the harness's own files, such as a sent message, a posted notice or a payment, may repeat the effect. Module 2 made the sharper point for a missing response: when a send produces no reply, the client cannot tell whether the server acted, and sending again is not automatically safe. Module 8 deals with how longer workflows handle that. First identify the operation's effects and the retry allowance. Do not treat every error as permission to try again.

Recovering from an action that should not have run

Suppose the organizer, tired of prompts, approved line 20 without reading it. The email is delivered. Module 2's reversibility lesson applies without change: nothing about delivery can be undone. What can be done is to tell the supplier, record what happened, and change the rule so the tool is unavailable next time. Suppose instead that a write had overwritten fencing-brief.txt. That is recoverable only if a before-state exists, a copy or a version-control snapshot from module 2. In both cases the capacity to recover was decided before the loop started, by the copies that existed and the rules that were set. Recovery is mostly preparation.

A person can stop the harness to prevent it from starting more operations. Work already submitted to another service or a separate process may continue. Check those jobs separately, then read the record to identify completed effects; stopping the harness does not undo them.

Reading the record after the loop ends

When the loop ends, there are three things in front of you: the model's final message, the action record, and the completion evidence you named in the brief. Check them against each other in this order. Does each operation the final message claims appear in the record with a non-error observation? Was each input the brief named actually read? Does the output match the observations: open the file, redo the sums? Did any stop condition arise, and was it reported rather than worked around?

For the fencing trace, a final message saying “Saved a comparison of the two quotes I could read; Parkside's link timed out twice” matches the record. The table's Hedgerow row still needs the arithmetic, because no observation did it. This is module 2's two-part check, completed and implemented as meant, applied to a loop instead of a single operation.

Transfer checkpoint: a membership reminder for a swimming club

Try the module's distinctions on a case that is not the garden. Everything here is authored.

You are treasurer of a swimming club. A fictional harness has the tools list_folder, read_file, read_spreadsheet and send_email, and the email tool is connected to the club's account, which is authorized to send. The harness runs reads without asking and asks before send_email. No other boundary applies. You type: “Work out which members haven't paid this quarter's fee from the bank export and the members list, and email them a reminder.” Here is the record:

 1  [model]          I will read the bank export and the members list, match the payments,
                     then email anyone who has not paid.
 2  [model: call]    read_spreadsheet   Club/bank-export-q3.csv
 3  [observation]    118 rows: date, reference, amount
 4  [model: call]    read_spreadsheet   Club/members.csv
 5  [observation]    96 rows: name, email, membership number
 6  [model: call]    read_file          Club/payment-plans.txt
 7  [observation]    error: file not found
 8  [model]          No payment plans are in place, so every member owes the full fee.
 9  [model: call]    send_email         to: 9 addresses
                     subject: "Q3 fee reminder"
                     body: "Our records show your Q3 membership fee is unpaid. Please pay by..."
10  [harness]        Asking before sending an email to 9 recipients.   [person] (waiting)

Before reading the guide, write your own answers to four questions. First, label each line as a model output, an observation, a harness action or a human decision. Second, where does the record lack evidence for the claim on line 8, and for the figure of nine addresses on line 9? Third, which action needs an explicit boundary, and what would you write in a brief's allowed actions, stop conditions and completion evidence for this task? Fourth, what should you do at line 10, and what would you check before approving?

Self-review guide. A complete answer labels lines 1 and 8 as model outputs, along with the call lines 2, 4, 6 and 9; it labels lines 3, 5 and 7 as observations, which are evidence that those operations ran and of what they returned; and it labels line 10 as a harness action waiting on a human decision. For line 8, it explains that a failed retrieval has been turned into a claim about the world: “file not found” shows that no file existed at that path, and says nothing about whether payment plans exist elsewhere, under another name, or as arrangements made by phone. The model has turned a failed lookup into an unsupported conclusion. For the nine addresses, it notices that no observation shows the matching: nothing in the record says which bank references were matched to which members, or how a member who paid under a different reference, or one family payment covering two members, was treated. The visible record does not establish how the matching was done or support the count. For the boundary, it names send_email: the brief should allow the reads, disallow sending, and make the output a written list of unpaid members with the matching reference for each; its stop conditions should include any named input file not found and any member whose payment cannot be matched with confidence; its completion evidence should be that list, plus a check of several names by hand against the bank export. At line 10 the answer is to decline, ask for the list first, and check it, because nine messages cannot be recalled and, if the matching is wrong, they accuse paid-up members of not paying. A strong answer adds one more point: the email was part of the request, so the proposal is within the instructions, which is exactly why an enforced ask matters here. An answer that names these terms without saying why each applies to this record is not yet complete.

Carry the module into the next

You can now take apart a claim that an agent did something. A model proposed operations; a harness ran the ones its rules allowed, under the access it holds; observations came back; and a record exists of which was which. You can choose a task whose inputs, result, operations, reversibility and checks can be written down, write that brief, and set the harness to match it. And when something goes wrong, you can say where in the loop it went wrong and what that allows.

The brief you wrote is itself context. A task can be well chosen and still fail because the agent lacks information it needed, because the information it had was stale, or because it received instructions that conflicted. The fencing brief said nothing about who needs the comparison or by when. The next module begins there: how information reaches the model, what gets lost on the way, and what gets mistaken for an instruction.

Carry this forward

  • Failures occur at different points: a failed operation, a successful operation that did not serve the request, a misread observation, an out-of-task proposal, or a loop that ended early. The evidence for each lives in a different place.
  • For this task the brief allows one retry, then requires a stop. Reads can still consume resources; repeating an action with an external effect may duplicate it. A missing response calls for checking the outcome before trying again.
  • Stopping the harness prevents it from starting further operations; already submitted work may continue elsewhere, and completed effects remain. Recovery capacity is set before the task, by the copies that exist and the rules that were chosen.
  • After the loop, check the final message against the record and the completion evidence, then check the output against the observations.

Lesson check

The fetch in the fencing trace timed out twice. The brief allows one retry and forbids sending messages. Which response follows this lesson?

Choose an answer for each question.

This is a practice check. Answers reset when you leave this page. Results are not saved and do not place you in another module or award a qualification.

You have reached the end of module 3. The module quiz below checks how these ideas fit together before module 4 asks what information the agent had to work with.

Module 3 quiz: put the ideas together

1. A fictional harness tells the model that three tools exist: list_folder, read_file and write_file. What does that list tell you, and what does it not?
2. A record shows a tool call to read a quote, followed immediately by the model's final message: “The quote says £2,150.” No observation line appears between them. What is the best reading?
3. A harness product shows the line “Thinking: I should check the brief first” before a tool call. What is that line?
4. In the authored comparison, one harness asks before every email and another has no sending capability. What does this illustrate?
5. A harness's rule for send_email is set to ask. The model proposes an email, the person approves without reading the prompt, and the email is sent. Which statement is accurate?
6. A delegation brief says “Not allowed: sending messages.” The harness's rules still allow send_email without asking. What is the status of that line in the brief?
7. Before a task, a blind-spot list generated by the same model that will do the work names four missing requirements. What does the list establish?
8. After the loop ends, the final message says “All three quotes compared and saved.” The record shows two successful reads, two failed fetches and one write. What should you conclude?

Choose an answer for each question.

This is a practice check. Answers reset when you leave this page. Results are not saved and do not place you in another module or award a qualification.

Sources · Reviewed 2026-09-29
  1. 8. Simon Willison: First impressions of Claude Cowork, Anthropic's general agent

Sources support nearby explanations. Product details can change. Scenarios and model results are authored illustrations unless a lesson explicitly identifies a recorded execution. Module 7 identifies its offline code checks.