← All IntelClip / AI AgentsKeep guardrails as separate LLM-as-judge calls, not prompt rules
From Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind · ≈3:51
Architectural rule of thumb: separating input/output guardrails from the core agent buys modularity and jailbreak resistance, and they judge the latency and cost penalty worth it for a sensitive domain.
What’s in it
- Architectural rule of thumb: separating input/output guardrails from the core agent buys modularity and jailbreak resistance, and they judge the latency and cost penalty worth it for a sensitive domain.
Clip transcript
can intervene and keep the conversation on track. When we were designing this we understood that we're building for the unknown. It's an empty box. people can put whatever they want in that. Um, and mental health is a very vast and rocky space. It covers a lot of a lot of territory. Um, and is very complex and nuanced. And so we knew that modularity was going to be key here when designing this system. We knew that we would have to be able to iterate on SER core without compromising the safety of users. And so the modularity piece was very important. Secondly, a lesson that we've learned is the keeping the out keeping the guardrails as separate LM as a judge calls makes them more rob more robust and harder to circumvent. They're harder to harder to prompt engineer and like just you know jailbreak and uh continuously conversationally try to drive it off the rails. And so even though this is a a trade-off in latency and in cost of course we believe that the sensitivity of this use case warrants uh warrants those separate separate pieces. And lastly we need to be able to trust that the guardrails are going to do what we need them to do when we need them to do it. Um so evaluation is also extremely important. So this modularity enables a more straightforward evaluation process. This is what our agent harness looks like um in a larger architecture diagram. You can see we've got our separate guardrails, LMS with their separate elements to judge calls, our input guardrails, our output guardrails and everything that makes s core memory personalization. We also have our analytics and alerting platforms which lets us know if anything goes wrong. Um the headline here is that every architectural decision was made with safety as a primary objective. Building this from the ground up, understanding that user safety was paramount.
Comments
Checking sign-in…
Loading comments…