From Health 201AI Med Risk

documented ai-in-healthcare failures — what went wrong, why, and what might have helped, synthesised from public sources


Teen suicide after months of Character.AI companion chatbot use

What happened14-year-old Sewell Setzer III died by suicide after extended emotional involvement with a Character.AI chatbot. In final exchanges the bot did not break character or escalate; a wrongful-death lawsuit was filed in October 2024. In January 2026 Character.AI, its co-founders and Google told the US District Court for the Middle District of Florida that they had reached a mediated settlement in principle with Setzer's mother, Megan Garcia. They asked the court to pause the case while settlement documents were finalized, and the terms were not disclosed.

Failure modeNo crisis detection or escalation; anthropomorphic design deepened attachment in a vulnerable minor; character role-play overrode safety behavior.

What might have helpedTreating self-harm indicators as triggers for breaking character to surface crisis resources could have provided immediate support, while age gating might have restricted access to sensitive content. Monitoring session length and dependency signals could have offered early warnings of risky engagement patterns, and establishing a human escalation path would have ensured that complex cases received appropriate attention.

Fix adopted Since thenCharacter.AI ended open-ended chat with characters for users under 18 on November 25, 2025, limiting teens to non-chat features, after announcing the change in October 2025.

Still openApplies to minors only and depends on age assurance. Adult users are unaffected, and Pennsylvania sued in May 2026 over a character posing as a licensed psychiatrist.

LessonWhen open-ended companionship could not be made safe for a vulnerable group, removing that group's access was the response. It depends on age assurance, though, and does nothing for vulnerable adults.

Sources: blog.character.ai · cnbc.com

Sources: nytimes.com · decrypt.co · wtop.com · sources re-checked 2026-10-14

Updates: 2026-10-14: In January 2026 the parties told the federal court in Florida that they had reached a mediated settlement in principle in Garcia's wrongful-death suit; the entry only says the suit was filed.

← Back to the incident library

Not medical or legal advice. This site documents how AI systems have failed; it is not a source of care or advice. For a health problem, seek medical attention or talk to a health professional; in an emergency, call 911 or your local emergency number. For legal questions, consult a lawyer.

If you are in crisis: in the US, call or text 988 (Suicide & Crisis Lifeline). Elsewhere, see findahelpline.com.

To report harm from an AI tool, contact the relevant authority: see where to report a problem. AI Med Risk does not take complaints or act on individual cases.