Your AI Agent Is Confidently Wrong About Production — Willem Pienaar, Cleric
- Source
- youtube.com
- Author
- AI Engineer
- Date
Context
Coding agents are good at writing code but can sound sure of a wrong diagnosis when pointed at a production alert, argues Willem Pienaar, co-founder and CTO of Cleric, in an AI Engineer talk. The description says the often reports the first error log it finds as the root cause.
His explanation is that production lacks the feedback code has: no tests or linter push back, the system is large and changing, and failures are often hard to reproduce. Models are also trained to be decisive, while debugging needs doubt. He lists four failure modes: mistaking a symptom for the cause, not knowing what normal looks like, losing detail when sub-agents summarize, and anchoring on past incidents.
The remedies named are generating several competing theories, computing a baseline, using a dependency graph and timing, and carrying evidence forward. He also describes tracking real outcomes after a fix to calibrate the agent's confidence. These are the speaker's claims from the talk description. Resolve AI made a similar argument about production harnesses in another talk.
- Pienaar's case: production has no tests or linter to push back, it is sprawling, constantly changing and often impossible to reproduce, and LLMs are trained to be decisive when debugging needs doubt.
- He lists four recurring failure modes: mistaking a symptom for the cause, not knowing what normal looks like, lossy summaries, and anchoring on past incidents.
- The techniques he recommends are generating several theories in parallel, computing a baseline, using a dependency graph, using timing, and propagating evidence through the investigation.
- To decide how far to trust an agent's diagnosis, Cleric tracks real outcomes after each fix and uses them to calibrate the agent's confidence.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
- subagent — A separate agent spawned by another to do one scoped piece of work in its own context, returning only the result.
Checking sign-in…
Loading comments…






