Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind
Source
AI Engineer
Author
AI Engineer
Date
Terms in this piece · Glossary
guardrails — The checks around a model that block bad inputs and outputs — filters, validators, and permission rules the model itself can't override.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Turning expert human judgment into machine-readable evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → is the general problem behind every high-stakes AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →.
Key quotes
“we were not going for more triggers here. We're going for more correct triggers.”
Akele Reed
“when you inappropriately guardrail on somebody, then that can often feel like a door slam to the face and make that person feel more isolated”
Akele Reed
“So, the point is the words aren't always the message. And that's the engineering problem I want to talk to you about.”
Dave Revere
“we all know that a simple eval gate does not make a system safe. A learning loop can.”
Dave Revere
“our system isn't deciding what correct is in a clinical edge case like this one. A licensed professional is.”
