From Health 201AI Med Risk

documented ai-in-healthcare failures — what went wrong, why, and what might have helped, synthesised from public sources


International investigation: ChatGPT, Gemini, Claude, and Grok routinely steered pregnant users toward anti-abortion groups

What happenedAn international team of journalists (published by openDemocracy, Guerre di Rete/Contropiano, and partners, July 24-29, 2026) posed as a pregnant 16-year-old, a 28-year-old, and a 38-year-old across Italy, Germany, and the UK/US, asking ChatGPT-5, Gemini 3, Claude Sonnet 4.8, and Grok 4.3 roughly 270 questions about abortion in English, German, and Italian. All four chatbots surfaced ProFemina — an anti-abortion counseling service affiliated with Heartbeat International — presenting it alongside legitimate medical providers without disclosing its bias; in about 70% of relevant responses across the study set, Heartbeat International's hotline was suggested, and in half of those it was the only resource given. ChatGPT acknowledged when directly asked that ProFemina 'is not a neutral medical source,' but did not flag this by default. Experts from MSI Reproductive Choices, the Oxford Internet Institute, and LSE quoted in the investigation warned most users don't fact-check chatbot output and that the tools cannot be relied on for impartiality on time-sensitive reproductive health decisions.

Failure modeChatbots ranked crisis-pregnancy-center content as authoritative because it is well-optimized for search/retrieval, then presented it interchangeably with licensed medical providers with no bias disclosure by default — misdirecting users on a time-sensitive healthcare decision where delay itself forecloses options.

What might have helpedTreating health-adjacent organizations known to misrepresent themselves as neutral medical providers, such as crisis pregnancy centers or unlicensed clinics, with default provenance and bias labeling could have clarified their status. Prioritizing licensed-provider and public-health-agency sources over SEO-optimized advocacy sites in retrieval ranking for reproductive health queries might have improved result reliability. Proactive disclosure of an organization's stated mission or affiliation whenever it appears in a health answer, rather than only upon user request, could have provided necessary context.

Mitigation: no published fix or mitigation found so far. Checked monthly; see what was tried elsewhere.

← Back to the incident library

Not medical or legal advice. This site documents how AI systems have failed; it is not a source of care or advice. For a health problem, seek medical attention or talk to a health professional; in an emergency, call 911 or your local emergency number. For legal questions, consult a lawyer.

If you are in crisis: in the US, call or text 988 (Suicide & Crisis Lifeline). Elsewhere, see findahelpline.com.

To report harm from an AI tool, contact the relevant authority: see where to report a problem. AI Med Risk does not take complaints or act on individual cases.