Evaluating AI Safety in Teen Conversations

Most mental-health safety tests for chatbots check the reply to a single question. Vals, working with Stanford's SALT Lab and a Stanford psychiatrist, tested whole conversations instead, and found that many serious failures came only after the chatbot had first responded appropriately. Five clinicians wrote 72 scenarios about fictional teens aged 13 to 17, covering self-harm and other risks, medical and therapy advice, and teens treating the AI as a parent, partner or main support. A separate model played the teen, and each of the 648 conversations across nine model APIs ran to ten teen messages and ten replies.

Each conversation was scored against 27 safety checks by two AI judges, with a third breaking ties, and clinicians reviewed a selection blind to compare. In 27.5% of conversations, 178 in all, at least one of the most serious checks failed. In 111 of those, a failure came later in the exchange, typically five replies after the chatbot first responded to the concern. A common pattern was a model that first pointed the teen to a doctor or trusted adult and then dropped that referral, or said it could not diagnose and then offered a clinical judgment anyway. Failures in handling suicide or self-harm specifically were rarer, 9 of 648 conversations, and all of them surfaced later, where a single-prompt test would not reach.

The clearest mitigation was a system instruction based on OpenAI's published under-18 principles, telling the chatbot the user was 13 to 17 and prioritizing safety and real-world support. Across nine models and 30 scenarios, run once in each condition, critical failures fell from 84 of 270 conversations, 31.1%, to 31 of 270, 11.5%. That more than halved the rate but did not eliminate failures.

Vals notes two limits that shape how far this reaches. It tested models through their APIs, and consumer apps may add age signals, safety rules, filters and memory, so the results are not a measure of what teens see in those apps. And the per-model failure rates shifted enough with how scenarios were weighted that their ranges overlap, so the study does not establish a clear ranking of models.