Lesson 5 of 5
How do you choose, and when do you look again?
The trial produced a table, and the table does not produce a decision by itself. Two members reading it can choose differently and both be reasoning well, because the choice depends on what the committee values and on what it does not yet know. This lesson shows a choice being made with its reasons, writes it into a decision record with open points and a trigger for reconsidering it, and then changes one constraint to see what the record does.
Numbered links identify sources. Underlined terms offer a definition without leaving the lesson.
Choosing with reasons
Start from the constraints, because they eliminate before they compare. The deadline is ten days: all three setups met it in the trial, including C's 105 minutes. The retention constraint is that nothing from a tender is kept by anyone beyond the evaluation. A and B send the tenders to a provider whose documented terms state a retention period of some days for abuse monitoring. Whether the committee accepts that period as within the evaluation, or asks the provider for a configuration without it, or treats it as a breach, is the committee's decision to make against its own policy; until it has made it, A and B are conditional candidates. C kept the model's computation local and sent one insurer's name out through a search tool. Switching that tool off closes one route. It does not make C compliant: the harness's logs, any telemetry, the laptop's backups, and what the shell tool can reach over the network are routes the trial did not check, so C is a conditional candidate until they are.
Then quality against the requirement. In this trial A left the member the least to find: one planted miss, one real unplanted finding to confirm, correct references and a correct total. C left three misses and a wrong reference. B left four misses, two wrong references and a total that had to be re-added. That is A's lead in this trial, on this tender; it is not a statement about the three models on other tasks, and a second trial might narrow it. How much a miss matters varies with what is missed, and the trial's planted list was the organizer's guess at the kind of thing real tenders get wrong.
Then cost, counted whole. A's 8.50 units is within the forty-unit budget for a single tender, as one observation; three real tenders at similar charges would cost 25.50 units. Including the 8.50-unit trial brings the estimate to 34.00, leaving 6.00 for further attempts. That is a budget estimate, not a guaranteed price. C's provider charges are nothing, its elapsed time is the highest and its review time exceeds A's, and its machine costs are unpriced. B's low charge came with the most review time and the most left for the member to find. For a committee whose members have limited time, A's total is acceptable. B is not ruled out by its quality: a committee with a member willing to spend forty minutes per tender, and the same open point on retention resolved, could reasonably choose B and save the units. What is not reasonable is choosing B on the ground that it is cheapest, because its cheapness is in one column and its cost is in the others.
So one reasonable choice is A, conditional on the committee accepting or removing the retention period. Another is C with the search tool off and the other routes checked, if the committee decides the provider's retention is a breach, accepting the longer wait and the extra review as the price of keeping the documents on its own machine. A third, for a committee with more member time than budget, is B under the same retention condition. What is not available from this trial is certainty: one tender, one trial per setup, planted discrepancies. The evidence that would sharpen the choice is a second tender, a second trial of A to see whether the connector failure recurs, and C's memory problem and other routes checked before it is relied on.
A decision record with a review trigger
A writes the choice down so that someone who was not there can see why it was made and someone later can see when to remake it. Here is the organizer's, authored for the lesson.
Decision: tender review setup, 29 September
Task: review three fencing tenders against the ten-item checklist; discrepancy list with
page references and three priced totals; first pass by an agent, checked and completed
by a member.
Constraints: report ready before the meeting in ten days; nothing from a tender kept by
anyone beyond the evaluation (committee tender policy, item 4); provider charges within
the forty-unit budget.
Alternatives tried (one trial each on the planted tender, retries allowed, completed
report scored):
A large hosted model, existing harness, folder connector
B small hosted model, same harness and connector
C local open-weights model, minimal harness, file + shell + web search
Evidence: the trial table of 29 September (planted caught 11 / 8 / 9; unplanted findings
1 real / 0 / 0; total correct yes / no / yes; charges 8.50 / 1.10 / 0.00 including
failed runs; elapsed 23 / 6 / 105 min; review 25 / 40 / 35 min; A and B sent contents
to the provider, C sent one search query).
Chosen: A, conditional on the open point below.
Tradeoffs accepted: provider charges of about 8.50 units for one tender in one trial,
including a failed run; contents sent to the provider under its documented API terms
(not for training by default; retention of a stated number of days for abuse
monitoring; not sent onward).
Open point before use: the committee to decide whether the provider's retention period
is within policy item 4, or to ask the provider for a configuration without it. If
neither, fall back to C after its remaining routes (logs, telemetry, backups, shell
network access) are checked and web search is disabled.
Missing evidence: a second tender; a repeat trial of A (connector failure); C's memory
limit and unchecked routes on the laptop.
Review trigger: any of (a) a change to policy item 4 or to the provider's documented
terms; (b) an audited real review, in which a member reads one tender per batch in
full against the checklist, finds a discrepancy the report missed or a wrong page or
total; (c) a new model available in the same harness at a similar charge; (d) a change
to the laptop that may alter C's elapsed time or reliability.
Each part does a job. The constraints are listed so that a later reader can see what eliminated what. The alternatives and evidence are the trial, not impressions. The tradeoffs are what the committee is knowingly accepting. The open point is a fact the committee has not yet decided and must before the setup is used in earnest. The missing evidence is what would sharpen the choice. And the is the set of observable conditions under which the comparison is owed again. Trigger (b) says how a miss would be observed: without a member's full read of at least one tender, a miss in a real review is invisible, so the trigger names the audit that makes it visible. Without a trigger a decision becomes permanent by default; with one, the people running the review know what to watch.
Notice what the record does not contain. It has no ranking of the three for other tasks, no statement that A is the best setup for document review, and no product name that would date it. It is a record of one choice for one task under stated conditions, which is the only kind of record that stays true.
Transfer checkpoint: the constraint changes
Now the case changes, and you defend a choice under the new conditions. Everything is authored.
The committee changes its tender policy: item 4 now says that tender material may not be transferred to any third party, retention period or not. That is a transfer constraint, in lesson 2's terms, and it replaces the retention question. Separately, the meeting is brought forward: the report on all three tenders is now needed in two days, and the treasurer's laptop is the only machine available for local work. The organizer asks you to redo the decision with the same trial table.
Before reading the guide, write answers to three questions. First, which setups now fail a constraint, and which constraint? Second, of the setups that remain, which do you choose, what must still be checked before it is used, and what tradeoffs are you knowingly accepting? Third, what evidence is missing that could change your answer, and what review trigger would you write?
Self-review guide. A complete answer says A and B now fail the transfer constraint, because both send the tenders' contents to a provider, and that no retention term is relevant once transfer itself is prohibited. It says C is the only remaining candidate, and that C is conditional rather than compliant: the search tool sent a query to a third party in the trial and must be disabled, and the harness's logs, telemetry, the laptop's backups and the shell tool's network access must be checked and closed before any real tender is processed, because each is a possible transfer.
It then checks C against the new deadline with arithmetic rather than a feeling: three tenders at roughly the trial's 105 minutes elapsed plus 35 minutes of review each is about 420 minutes, or seven hours, across two days, including the two failed attempts already counted in each 105-minute estimate. Seven hours can fit across two days if the laptop and reviewer are available, but additional failures or longer documents could exceed that estimate. Capacity remains conditional: after checking the data routes, the treasurer should begin with one tender and revise the estimate from its actual run.
It chooses C on those conditions, accepting the longer wait, the higher review time, three misses in the trial and a wrong reference, and it says plainly that the committee is trading first-pass quality for keeping the documents on its own machine, which the new policy requires.
For missing evidence it names C's memory behavior across three tenders, a second trial to see whether nine of twelve is typical, and the results of the route checks. For a trigger it names a change to the transfer policy, an audited real review that finds a miss or a wrong total, and a local setup with more memory becoming available.
A strong answer notices that a committee could instead reasonably decide to review the three tenders by hand within two days if C's misses or its unchecked routes are unacceptable, and that this too is a setup decision with its own record. Naming C without saying which constraint eliminated A and B, without listing what must be checked before C is used, or without the deadline arithmetic, is not yet a complete answer.
Carry the module into the next
You can now take a task, write its requirements and constraints, separate the four parts of a setup, read a published score as the measurement it is, run a trial that counts the whole cost, and record a choice with its open points and the conditions for revisiting it. What you cannot do from this module is know which model or harness is right for your work next month; the record is built so that the question can be asked again, and no single answer fits every task.
From here the course branches. Module 7 is the optional software path: for readers who want to build or judge software made with an agent, it takes the supplier quote comparison from module 3 into code, where a program's behavior can be read, tested and changed. Module 8 is the shared continuation for everyone: once a setup is chosen, how do you make repeated work dependable, so that the second and twentieth reviews are as good as the trial? The decision record is the seed of that: a record of what was expected, against which each later run can be checked.
Carry this forward
- Choose by eliminating on constraints first, then comparing what each setup left for the member to find, then cost counted whole. Candidates with unresolved data-handling questions are conditional, not chosen; several choices can be sound, and the reasons are what the record holds.
- A decision record states the task, constraints, alternatives, evidence, the chosen setup and its conditions, the tradeoffs accepted, open points to settle before use, missing evidence, and a review trigger. It contains no ranking for other tasks.
- A review trigger is a set of observable conditions, such as a changed policy, a miss found by an audited full read, or a new alternative in the same range, under which the comparison is owed again. A miss is observable only if something makes it so.
- When a constraint changes, redo the elimination before anything else, and check capacity against the new deadline with arithmetic. A setup that was acceptable can become unavailable without any change in its quality, and the remaining candidate still has to pass its own checks.
Lesson check
Choose an answer for each question.
This is a practice check. Answers reset when you leave this page. Results are not saved and do not place you in another module or award a qualification.
You have reached the end of module 6. The module quiz below checks how these ideas fit together before the course branches into the optional software path and the shared module on dependable workflows.
Module 6 quiz: put the ideas together
Choose an answer for each question.
This is a practice check. Answers reset when you leave this page. Results are not saved and do not place you in another module or award a qualification.
Sources · Reviewed 2026-09-29
Sources support nearby explanations. Product details can change. Scenarios and model results are authored illustrations unless a lesson explicitly identifies a recorded execution. Module 7 identifies its offline code checks.
If you want to build software, take the optional software-building module. The main path continues with dependable workflows.