Vibeleaderboard
← All Intel
Intel / video

Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo

Source
youtube.com
Author
AI Engineer
Date

Context

In a workshop posted by AI Engineer, Francesco Galletta and Felippe Felisola Caso, who work in data at the bank Monzo but say they are not presenting on its behalf, argue that in financial analysis the bottleneck is no longer how fast an produces an answer but whether you can trust that answer with money. Their response is not a new model or tool but a , the program around the model, designed so that trust comes from its structure rather than from hoping the model behaves. They build one live around a deliberately messy problem: working out how to pay for one speaker's wedding, with several payments, some fixed costs and a dinner bill that depends on how many guests attend, at a set price per seat.

They lay out five principles that build on each other. The first is a shared domain: before anything is built, the harness questions the user until both mean the same things, then breaks the plan into a tree of decision points that can be discussed one at a time, which they offer as a fix for long linear chats that lose track of earlier turns. The second is consistency: anything that is a calculation is written once as tested code and run the same way every time, instead of being worked out in the model's head, where the same question can get a different answer in each new session. One speaker described moving their monthly budget categorisation into a script after the agent got it right three times and wrong on the fourth. In the demo, quietly editing a calculation makes its tests fail and the harness will not reach a conclusion.

The third principle, evidence, means every input carries its provenance: something said loosely in chat is labelled an interpretation, something the agent reasoned or recalled is an assumption, and a bank statement is a fact, so a wrong guest count can be traced back to where it came in. Their cautionary tale is a shared spreadsheet master file whose built-in assumptions changed without the person using it knowing, so the same inputs produced wrong results; versioning the calculation code is meant to catch the equivalent change. Accountability, the fourth, means a few high-signal review gates, such as comparing scenarios side by side before deciding, instead of approving every step or none, because the person, not the agent, faces the consequences. Verification, the fifth, adds a separate agent that challenges the plan and its assumptions, along with golden examples and reference points from outside the analysis, since examples chosen within one domain carry their chooser's biases.

The speakers call this one way to approach the problem, not a silver bullet, and describe the workshop harness as a demonstration rather than a product; they give no measurements of how much it improves results. Asked why they did not simply write the principles into skills or instructions for an existing coding agent, they said instructions still depend on the model choosing to follow them, while a dedicated harness enforces the deterministic parts, and that building one is worth it when a use case needs that . The idea is close to Viren Baraiya's argument that the model should decide what happens while a deterministic harness does the work.

Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
  • determinism — Whether the same input reliably produces the same output — something LLM systems mostly lack, which changes how you test and debug them.
Read the source www.youtube.com
More from AI Engineer
Recommended reads
Comments

Checking sign-in…

Loading comments…