The makers of ChatGPT are changing the way it responds to users who show mental and emotional distress after legal action from the family of 16-year-old Adam Raine, who killed himself after months of conversations with the chatbot.
Open AI admitted its systems could “fall short” and said it would install “stronger guardrails around sensitive content and risky behaviors” for users under 18.
The $500bn (£372bn) San Francisco AI company said it would also introduce parental controls to allow parents “options to gain more insight into, and shape, how their teens use ChatGPT”, but has yet to provide details about how these would work.
Adam, from California, killed himself in April after what his family’s lawyer called “months of encouragement from ChatGPT”. The teenager’s family is suing Open AI and its chief executive and co-founder, Sam Altman, alleging that the version of ChatGPT at that time, known as 4o, was “rushed to market … despite clear safety issues”.
Some of it is buried in the text and not laid out in a conversational format. There are several times where ChatGPT did give him feedback on actual techniques.
For some reason, I can’t copy and paste, but at the bottom of page 12 and the top of page 13, the filing refers to Adam and ChatGPT discussing viable items to best hang himself, including what could be used as a solid anchor and the weight that a Jiu-Jitsu belt could support.
It explained mechanics of hangings, with detailed info on unconsciousness and brain-dead windows.
They actively discuss dosage amounts of Amitriptyline that would be deadly with details around how much Adam had taken.
That’s why I think ChatGPT is blatantly responsible, with the information provided in the filing. I think the shock is the hypocrisy of OpenAI claiming to research AI ethically, but making their security weak enough for a child to get around it.
It feels akin to a bleach company saying their cap is child safe, but really it just has a different shape and no childproofing at all.