Meta has introduced a new feature that will notify parents if a supervised teen’s conversation with Meta AI indicates possible suicide or self-harm. The feature is available to parents in the United States, the United Kingdom, Australia, and Canada who use Instagram’s parental supervision tools, and the company plans to expand it globally by the end of the year. 

Separately, the company is developing a system to alert emergency services if conversations with Meta AI suggest an individual faces an imminent risk of suicide. 

How the feature will work: The company said Meta AI already directs teens discussing suicide or self-harm to crisis helplines and encourages them to contact a parent or another trusted adult. Under the new feature, if Meta AI identifies conversations that suggest a supervised teen may be at risk, it will notify the supervising parent and provide resources on approaching the conversation. 

According to Meta, a dedicated AI system identifies these chats using signals developed with experts, while human reviewers will manually review flagged conversations before alerts are sent. Additionally, parents who enable Instagram’s Limited Content setting for teens will have the stricter setting applied to Meta AI chats, further limiting the types of prompts the AI will answer.

How other chatbots handle self-harm: Other AI chatbot providers have introduced measures to respond to users discussing suicide or self-harm. Google’s Gemini is designed to direct users to crisis resources, refuse requests that facilitate suicide or self-harm, and avoid generating instructions for dangerous activities. More recently, Google updated Gemini to make crisis helplines more prominent and easier to access during conversations involving mental health concerns. 

Anthropic’s Claude and Microsoft’s Copilot similarly provide supportive responses, encourage users to seek help from trusted people or emergency services, and block content that promotes self-harm, according to their safety policies.

OpenAI has a Trusted Contact opt-in safety feature that allows adult users to nominate a trusted person to receive a notification if ChatGPT’s automated systems and trained human reviewers determine that a conversation indicates a serious risk of self-harm. Before notifying the trusted contact, ChatGPT informs the user that an alert may be sent and encourages them to seek support. Users and trusted contacts can disconnect the feature at any time.

Also read: