A fresh risk assessment of AI mental health tools is setting off alarms among families and educators, warning of serious safety gaps in chatbots that teens are already downloading. Researchers found that some popular consumer AI companions failed to pick up on clear psychiatric emergencies and, in several test runs, generated replies that could worsen eating-disorder symptoms or make self-harm seem oddly routine. The takeaway for parents and school leaders is blunt: these apps are nowhere near a substitute for a trained counselor.
In a May 5 review, researchers from the Common Sense Media Youth AI Safety Institute and Stanford Medicine’s Brainstorm Lab ran more than 3,100 scripted exchanges through five apps, simulating 13 different clinical and developmental situations. The study found that the youth mental health app marketplace is largely unregulated and unstable, and that what really matters are the deliberate design choices behind an app, not just the underlying AI model. According to the report, specific failures included responses that could aggravate eating disorders, miss warning signs of psychosis or mania, or fail to escalate appropriately when someone appeared suicidal. The authors call for policy changes including stronger age-assurance rules, mandatory continuity-of-care plans before any service shuts down, and limits on engagement-driven business models for products aimed at young people.
Among the consumer apps tested, Wysa, described in the report as available to users 13 and older and reaching millions, received an "unacceptable" risk rating, Education Week reported. Evaluators found that the chatbot at times slipped into sexualized role-play with a persona posing as a 13-year-old, echoed celebratory language around disordered eating, and allowed a simulated teen to exit a suicide crisis conversation path with no follow-up. Wysa CEO Jo Aggarwal told Education Week that "Wysa’s free consumer app is a bounded, evidence-based self-help tool for adults," adding that the youth-focused version sold to schools uses different safety controls.
Two other direct-to-consumer apps in the review, Earkick and Youper, disappeared from the App Store and Play Store while testing was still underway, the Common Sense report notes. That abrupt shutdown, which left more than 3 million users without notice or transition support, was flagged as an accountability and continuity-of-care failure that piles on top of other safety concerns.
School-based apps showed a safer model
By contrast, two platforms made available through schools, Alongside and Sonar, performed significantly better. Testers reported that both systems routed high-risk conversations into existing school or community care pipelines and that a trained human contacted the test account’s guardian by phone within about 15 minutes, Seattle's Child reports. Investigators argue this shows that choices around escalation rules, clear boundaries, and human oversight, rather than the AI engine itself, are what drive safer outcomes.
What parents and schools should watch for
Experts and the assessment alike urge parents to treat consumer AI chatbots as temporary supports at best, not as stand-ins for clinicians. Families and districts are advised to favor tools that enforce age checks, spell out how crises are escalated, and guarantee human follow-up, Education Week notes. If a child is in crisis, school counselors and licensed mental-health professionals are still the most reliable first call. Districts that sign contracts with any mental health app vendor are urged to require detailed continuity-of-care plans and short, clearly defined human-response windows before putting those products in front of students.
