Whisper transcription tool fabricating text in medical settings
What happenedAP investigation found OpenAI's Whisper — used by medical transcription vendors in clinics — fabricates sentences, including invented medications and racial commentary, sometimes deleting the source audio after transcription.
Failure modeHallucination in silence/gaps; deployment in clinical documentation despite explicit vendor warnings against high-risk use; audio deletion removed the verification path.
What might have helpedRetaining source audio and applying confidence flagging to low-signal segments could have provided a basis for verification. Having clinicians review transcripts before chart entry might have caught errors that automated systems missed. One safeguard would be to avoid deploying general-purpose automatic speech recognition in clinical pipelines without specific domain validation, as this approach could have reduced the risk of misinterpretation.
Mitigation: no published fix or mitigation found so far. Checked monthly; see what was tried elsewhere.