Vibeleaderboard
← All Intel
Intel / video

Shipping AI to a Million Patients Without an A/B Test — Jared Joselowitz, Ufonia

Source
youtube.com
Author
AI Engineer
Date

Context

Ufonia describes testing its patient-calling voice against simulated conversations before exposing patients to a changed system. In its evaluation presentation, clinicians help specify potential hazards, one model plays the patient and another evaluates the resulting dialogue.

The proposed testing loop also checks the simulated patients and automated evaluator against human assessments. That matters because a convincing conversation or favorable automated score is useful only if the test represents the failures clinicians want to catch. Simulation supplies repeatable cases for finding problems before deployment, while remaining one part of the evidence needed.

Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Read the source www.youtube.com
More from AI Engineer
Recommended reads
Comments

Checking sign-in…

Loading comments…