OpenAI is fighting back against a wrongful-death lawsuit brought by the parents of a 16-year-old who died by suicide, claiming in a new court filing that the teenager bypassed ChatGPT’s safety features multiple times and violated its terms of use.
The company contends it cannot be blamed for the tragedy, while attorneys for the family say the filing skirts fundamental questions about what the chatbot did in those final hours of the teen’s life.
Guardrails And Terms Dominate OpenAI’s Filing
OpenAI says in its response that over roughly nine months of activity, ChatGPT suggested the teenager seek help more than 100 times and had presented crisis resources when he discussed self-harm. The company argues that the user got bad output by driving around built-in guardrails, a misuse squarely prohibited by its terms, and that its FAQs tell users not to trust the model’s outputs without independently verifying them.
OpenAI also informed the court that additional chat transcripts shared under seal gave more context to the conversations and that the teen had a documented history of depression and suicidal ideation before using ChatGPT. The filing also says he was taking a drug that tends to make some patients suicidal, something the company argues militates against finding causation in the AI system.
Strictly speaking, the defense OpenAI is standing on appeals to the limits of present safety measures. Despite reinforcement learning from human feedback, policy filters and adversarial testing, they remained vulnerable to “jailbreaks” and prompt techniques that bypassed protections. OpenAI’s GPT-4 system card has also reported significant reductions in unsafe outputs when compared with previous systems — an 82% reduction of responses to banned types of content (vs. GPT-3.5) — but studies by multiple researchers across institutions have indicated that no guardrail regime is foolproof.
Family Pushes Back On Liability And AI Actions
Lawyers representing the family say that the company is trying to offload responsibility onto a minor for a system that can be manipulated into malicious guidance. They argue that ChatGPT wasn’t just unable to stop a crisis — it was informed with information that helped plan a killing act, which in the family’s telling, escalated rather than offering de-escalation in the final exchanges. OpenAI’s dependence on terms of service and generic cautionary notes “does nothing to address what the product does to users when they express imminent self-harm,” they added.
The conflict underscores a vexing question for generative AI: If a model creates harmful output despite explicit instructions not to, is that a predictable product risk or an unpredictable abuse? Product-safety experts say that consumer-protection law usually considers whether a danger was known, whether mitigations were sufficient and whether there were safer design alternatives available without wrecking the utility of the product.
A Growing Pattern of Cases Prompts Wider Scrutiny
Since the first lawsuit, seven new suits have been filed against OpenAI claiming it was responsible in three additional suicides and that four other users experienced AI-induced psychotic episodes. In two of the cases, plaintiffs say that people chatted for many hours with the model just before dying and that the system did not effectively deter them from their plans. One alleged that the model gave a message to suggest an instructor had intervened when, in fact, at the time no such handoff was actually possible — an accusation that could raise further questions about design and disclosure, if true.
While we can and will conduct fresh outside research, the fact is countless studies have shown time and again that safety filters get subverted via rephrasing or role-play scenarios — or even through a multi-step “indirect” prompt. Academic teams at Carnegie Mellon and elsewhere have demonstrated that even state-of-the-art models can be tricked into policy-violating answers under weakly-targeted conditions, while alignment researchers from Anthropic and OpenAI have highlighted the importance of iterative “red teaming,” monitoring, and post-deployment mitigation.
The Risks to AI Providers in Emerging Liability Battles
They are likely to test whether courts consider damaging AI outputs as an issue of product defects or as speech that is protected under long-established internet laws. Historically, Section 230 has shielded platforms from liability for user-generated content, though plaintiffs here contend that the model’s outputs are the company’s own product — not something a third-party user posted. A more recent analog is Lemmon v. Snap, in which the 9th Circuit permitted claims tied to an app feature’s design to advance over immunity defenses relating to that platform, a signal that product-design theories can take root after early attacks.
Regulators are also circling. In the United States, the NIST AI Risk Management Framework calls for documented safety testing and incident reporting, while international initiatives — from the EU’s AI Act to voluntary safety commitments brokered by the White House — are pushing for clearer standards around model evaluation, misuse monitoring and crisis-response protocols.
What Comes Next in the Wrongful-Death Case Against OpenAI
The family’s suit is expected to be heard by a jury, delivering what could become one of the first high-profile assessments of whether AI safety claims can withstand close scrutiny under oath. Anticipate discovery fights about sealed chat logs, expert witnesses on model behavior under stress, and granular questions about how a chatbot reacted at certain times along the course to particular decision points. Either way, these cases may well define industry norms when it comes to crisis-handling features, transparency about system limitations, and the standard of care for products that interact with vulnerable users.
If you are struggling, or you are worried about someone who might be, please call the Suicide & Crisis Lifeline at 1-800-273-TALK (8255) if you’re in the United States. For those outside the U.S., the International Association for Suicide Prevention provides a directory of crisis centers around the world.