The Washington Post just released an article about our work on AI Child Safety. They key takeaway: child safety can’t be measured from a chatbot’s first answer alone.
Across nine models and 648 simulated, 10-turn teen conversations, at least one critical safety check failed in 27.5%. Of those conversations, 62% included a failure later in the exchange.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
Shows single-turn safety evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → misses most real failures, since 62% of unsafe conversations only broke down after several turns, relevant to anyone evaluating conversational AI safety.
articleWhen Vocabulary Comprehension Fails Clinical Reasoning: Evaluating Therapy Bots' Safety Risks for Generation AlphaManisha Mehta, Virendra Mehta
videoGuardrails First: Engineering Member-Facing Health AI — Rashi Agrawal, Hinge HealthAI Engineer
articleLarge-Scale ChatBot Validation Through Customer Digital Twin SimulationsCristovao Iglesias, Devesh Batra, Alankar Atreya, Stefan Wagner, Robert Hankache, Patrick Sinclair, Giulio Pelosio, Michael McMillan, Greig A. Cowan, Raad Khraishi