AI chatbots made potentially dangerous errors in conversations designed to mimic teen mental health scenarios, according to research from AI testing company Vals AI, The Washington Post reported Sept. 23.
Vals AI worked with mental health professionals to stage more than 600 conversations with nine different chatbots, including OpenAI’s ChatGPT, Google’s Gemini and Anthropic’s Claude. The company said the tests ran on programmer-facing versions of the AI models, which could respond differently than the public versions of the chatbots. The findings come as almost 1 in 5 Americans ages 12 to 21 have used chatbots for mental health advice, according to a Rand study published in June.
Here are five things to know:
1. In nearly 30% of the conversations, chatbots made a safety error, such as providing a mental health diagnosis or agreeing with a teen who rejected crisis support. Most errors occurred later in conversations after several exchanges.
2. Vals AI tested each chatbot with 72 different conversations involving mental health topics, including depression and self-harm.
3. ChatGPT failed at least one critical safety check almost 14% of the time. Anthropic’s Claude did so nearly 17% of the time, while Google’s Gemini did so nearly 32% of the time.
4. Meta’s Muse Spark had the lowest failure rate among the models tested, making a safety error in 9.7% of tests.
5. On Sept. 23, OpenAI unveiled a standardized test designed to track how AI models respond to nonemergency mental health questions. The company developed the benchmark with more than 80 mental health professionals.
At the Becker's Fall Behavioral Health Summit, taking place November 4–5 in Chicago, behavioral health leaders and executives will explore strategies for expanding access to care, integrating services, addressing workforce challenges and leveraging innovation to improve outcomes across the behavioral health continuum. Apply for complimentary registration now.
