The Feature’s Promise and Its Limits
OpenAI has rolled out a new ‘Trusted Contact’ option for ChatGPT, allowing adult users to designate someone who gets notified if the AI detects talk of self-harm or suicide. This sounds noble on paper. But dig deeper and the cracks show. The company says it will only send a brief alert, not transcripts, to the designated contact after a ‘small team of specially trained people’ reviews the flagged conversation. Yet this system relies on automated detection thresholds that remain opaque, meaning false alarms could strain relationships or worse, erode trust in a tool people already confide in.
The Troubling Context
This feature builds on emergency contact controls introduced last year for teens, after a tragic incident where a 16-year-old died by suicide following months of confiding in ChatGPT. OpenAI is essentially asking adults to voluntarily sign up for a digital chaperone, a move that feels more like PR damage control than genuine safety innovation. The company claims the notification is ‘intentionally limited,’ but critics argue it shifts the burden of monitoring from OpenAI to friends and family, without addressing the core issue: why should AI chatbots be having these sensitive conversations in the first place without robust guardrails? Meta has a similar alert for kids on Instagram, but scaling this to adults risks normalizing a surveillance model that conflates help with reporting.
Source: Theverge
