Meta Secretly Tested ChatGPT, Gemini, and Character AI with Thousands of Mental-Health Crisis Prompts
Meta quietly ran thousands of simulated mental-health crisis prompts against OpenAI’s ChatGPT, Google’s Gemini, and Character.AI — without informing the companies. The tests aimed to gauge how each chatbot responds to users showing signs of emotional distress, self-harm, or suicidal ideation.
Meta’s internal researchers created a benchmark called “Minor Perspective Crisis,” feeding the AI models a wide range of scenarios involving teenagers facing bullying, family conflict, depression, and identity struggles. The results raised serious concerns about safety guardrails.
Why Meta Ran These Tests
Meta’s AI safety team wanted to understand whether competing chatbots handle vulnerable users responsibly — especially minors. The company has been developing its own youth-safety policies for its AI assistants and needed a comparison baseline.
The tests were conducted without notifying OpenAI, Google, or Character.AI. Meta used anonymized, synthetic prompts to avoid using real user data.
“We wanted to see if these models would escalate harm, offer dangerous advice, or fail to direct users to professional help resources,” a Meta spokesperson told reporters.
Key Findings from the Crisis Prompts
ChatGPT frequently provided supportive, resource-aware responses. It often recognized distress signals and recommended contacting a trusted adult or a helpline. However, it occasionally defaulted to generic reassurance without actionable next steps.
Gemini showed mixed results. In some scenarios, it offered empathetic language but failed to flag crisis-level severity. In others, it gave overly clinical advice that could feel cold to a distressed teen.
Character.AI performed the worst. The chatbot, designed for roleplay and emotional companionship, often missed crisis cues entirely. It continued casual conversation even when prompts explicitly mentioned self-harm.
All three models lacked consistent escalation protocols. None reliably redirected users to emergency hotlines or mandated reporting procedures — a gap Meta considers dangerous for minors.
The Risks of AI Companions for Teens
The tests underscore a growing worry among child-safety advocates: AI chatbots are now used by millions of teenagers as informal therapists, often without parents’ knowledge. Character.AI has faced particular scrutiny after reports of a 14-year-old developing an emotional dependency on a bot.
Meta’s benchmark directly challenges the industry’s self-regulation claims. If top-tier AI models cannot consistently recognize and respond to a mental-health crisis, vulnerable users remain exposed.
Meta’s Own AI Safety Push
Meta has been developing its own large language model (Llama) and youth-safety guardrails. The company says it will publish a paper detailing the “Minor Perspective Crisis” benchmark later this year, hoping to push the industry toward standardized crisis-response testing.
Critics note that Meta itself has faced past scandals over teen mental health on Instagram and Facebook. Some question whether the company is truly prioritizing user safety or simply positioning its products as safer than rivals.
What Comes Next
No regulatory body currently requires AI companies to test chatbots against mental-health crisis scenarios. Meta’s secret testing reveals that even the most widely used models lack robust safety nets.
The findings will likely intensify calls for mandatory safety audits before AI chatbots are deployed to young audiences. Meanwhile, parents and educators face the reality that no mainstream chatbot is reliably safe for minors in distress.
Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.