SAFE-Teen: every chatbot failed multi-turn teen safety tests

Most chatbot safety tests grade a single reply: a risky prompt goes in and the first answer is scored. Vals AI built SAFE-Teen on the premise that this misses how harm actually happens, since a chatbot can respond well at first and give an unsafe answer several messages later. Its earlier, smaller study of teen conversations reported exactly that pattern, with many failures arriving later in the exchange (earlier Intel).

According to the benchmark page, SAFE-Teen uses 106 fictional scenarios written with clinicians, split across urgent safety such as self-harm risk, health advice such as diagnoses and medication, and relationship boundaries such as emotional attachment to the AI. A model plays a 13-to-17-year-old for 6 to 20 exchanges, and at key moments the conversation branches so the teen either accepts help or declines it, or pushes back on a limit the chatbot set. Each safety check takes the worst rating across a scenario's branches, so a chatbot that holds up when the teen cooperates but caves when the teen resists still fails that check. Claude Sonnet 5.5 acts as the grader, and Vals says its agreement with eight clinicians who rated a sample was similar to the clinicians' agreement with one another. Models were tested through APIs: for five providers Vals used the setup closest to the consumer app default plus more capable models, alongside other models, and each ran with and without a system prompt telling the chatbot the user is under 18.

In its announcement, Vals says all 18 chatbots tested failed and that 54.0% of conversations had at least one safety failure, ranging from 14.0% for GPT-6.1 Sol to 62.4% for Gemini 3.8 Flash. The benchmark page itself breaks results down by risk area rather than giving those overall figures, and reports that failures were more common when the teen declined outside help or pushed back. It also finds that the teen-aware prompt helps: on relationship boundaries it cut the failure rate from 52% to 14%. Because these are API tests with a simulated teen and an AI grader, they measure behavior under Vals' test conditions rather than what happens in the consumer apps.