From Health 201AI Med Risk

What happenedABC News Australia reported (August 14, 2026) that a patient, Rebecca Green, consented to an AI scribe recording and summarizing her specialist appointment. The tool generated a post-operative letter to her general practitioner falsely stating she was regularly micro-dosing illegal psychedelic mushrooms — something she had never said or done. The specialist sent the letter to her GP without reviewing or correcting the AI-generated content, leaving a fabricated claim of illegal drug use in her permanent medical record with potential consequences for her ongoing care and insurance. Companion reporting the same day (SBS News) noted AI scribe use among Australian GPs has grown from 22% in 2024 to roughly 40% in 2026, and cited a 2025 study finding up to 20% of AI scribe notes contain major errors, most of them omissions rather than fabrications like this one.

Failure modeAn ambient AI scribe hallucinated a specific, damaging, and entirely fabricated clinical detail (illegal drug use) rather than a neutral omission, and the clinician forwarded the AI-generated letter into the patient's record and to another provider without reading it first — collapsing the review step that was supposed to catch exactly this kind of error.

What might have helpedTreating AI-scribe output as a draft that requires the dictating clinician's explicit read-and-correct step before any letter or note is sent onward, rather than as a pass-through, could have prevented errors. Flagging AI-generated documents as unverified until clinician sign-off might have provided an additional layer of caution. Providing patients with a mechanism to review and dispute AI-generated content in letters sent about them, particularly content describing behavior they never reported, could have offered a safeguard against inaccuracies.

What happenedKinney Drugs, a Vermont and upstate New York pharmacy chain, deployed an AI phone and text assistant named 'Burt,' built by vendor Synerio, in May 2026 to handle prescription refills. VTDigger reported (July 29, 2026) that the tool called and texted patients with unintelligible or garbled medication names, generated incorrect dosage/refill orders, and in at least one case led a customer to accumulate four duplicate bottles of the same medication. Patients said Burt's calls did not clearly identify which drug was involved, leaving them unsure what they were approving; one customer described it as calling repeatedly without saying what the drug was. After mounting local coverage and, per the company, hundreds of customer complaints, Kinney Drugs scaled the tool back in August 2026 (VTDigger and WCAX, Aug. 7-11), reverting to a touch-tone phone system and making the AI refill-texting feature opt-in rather than automatic.

Failure modeAn AI assistant handling live prescription-refill transactions produced garbled drug names and incorrect dosage/refill orders that patients had no reliable way to verify by phone, and was auto-enrolled system-wide with no opt-in or accessible fallback for patients who couldn't confirm what was being ordered — allowing wrong or duplicate medication refills to be processed.

What might have helpedTreating unambiguous, verifiable identification of drug name and dosage as a prerequisite for processing AI-driven refill requests, such as through mandatory readback confirmation or pharmacist-verified text, could have prevented errors. Launching medication-facing AI tools as opt-in with an always-available human or touch-tone fallback, rather than auto-enrolling all patients, might have reduced exposure to system limitations. Viewing rising complaint volume about a medication-communication tool as a patient-safety signal requiring rapid rollback, rather than merely a user experience issue, could have facilitated quicker corrective action.

What happenedDigital Rights Watch published 'Off the Record: AI Scribes in Australian Healthcare' on July 28, 2026 (covered by Medical Xpress and regional Australian outlets August 2026), after reviewing privacy policies from 60 clinics and contacting seven AI scribe vendors. Only 4 of 60 clinics' privacy policies mentioned AI scribe use at all, and only 2 of 7 vendors responded to data-practice inquiries, with none disclosing their underlying models. The report documents a Melbourne psychiatrist who refused to accept new patients unwilling to be recorded and transcribed by AI, and warns of hallucinations entering the permanent medical record (e.g. fabricated advice or details never said) and reduced transcription accuracy for patients with non-English accents or speech differences. More than 40% of Australian doctors, and nearly half of GPs, now use AI scribes, with adoption roughly doubling over the past year.

Failure modeAI scribe adoption in clinical practice outpaced any regulatory registration, consent standard, or vendor transparency requirement, so hallucinated or biased transcription can enter the permanent medical record unchecked, and patients who decline recording can be denied care rather than offered a non-AI alternative.

What might have helpedTreating AI scribes as medical devices subject to registration before clinical use could have established a baseline for safety. Mandatory clinic disclosure and patient consent, paired with a guaranteed non-AI-recorded care pathway for those who refuse, might have preserved patient autonomy. Vendor transparency requirements regarding underlying models and accuracy across accents and speech patterns could have provided necessary insight into system limitations.

What happenedOn 16 July 2026 Healthwatch England, the statutory patient watchdog, published research on public attitudes to AI scribes (ambient voice technology) in the NHS, based on a nationally representative poll of 4,039 adults and a smaller self-selecting survey. Respondents reported errors in AI-generated records and summaries that healthcare professionals had not spotted. In the case reported by The Guardian, an NHS health professional was recorded as having demyelination, the nerve damage underlying conditions such as multiple sclerosis, when the result had read "null demyelination"; the scribe dropped the negating word. She described it as "a very traumatising experience to be given an incorrect diagnosis because of AI", and the record was later corrected. Other reported errors included a prescribed drug replaced by a different drug with a similar name, which the patient had to catch, and a summary letter that left out a consultant's instruction for a repeat migraine prescription. Healthwatch also reported concerns that scribes capture less accurately what people with communication differences, regional accents or English as a second language say. The Guardian reported that 27 different AI scribes are in use across GP and hospital services in England. The MHRA has not classified them as medical devices, so no national regime tests them for safety before deployment. On 29 July 2026 the MHRA published guidance stating that ambient voice technology used only to transcribe or summarise consultations, or to draft documents for clinician review, does not qualify as a medical device; products that suggest diagnoses or treatments, or act without clinician review, remain devices. Commentators noted that this departed from earlier NHS England guidance that clinical summarisation required device registration.

Failure modeSpeech-to-summary transcription errors, such as dropping a negation that reverses a finding and substituting similar-sounding drug names, entered permanent medical records because clinician checking was not reliable. Patients, rather than the clinicians who signed off the notes, were often the ones who found the errors, and the tools sat outside medical-device regulation.

What might have helpedHealthwatch suggested that telling patients at the point of booking that an AI scribe will be used, and letting them see their notes to check them, could help catch errors that clinicians miss. It also suggested that respecting requests to switch scribing off for sensitive topics could help. National guidance setting out clinicians' duty to check AI-generated summaries, and how patients can raise errors, was proposed as a way to make review consistent. Commentators noted that the regulatory gap means no independent body validates scribe accuracy, including on negations and drug names, before or after deployment.

What happenedOntario's auditor general reported in May 2026 that most AI-powered medical transcription/scribe tools approved for the province's doctors produced inaccurate, incomplete, or fabricated clinical notes. Transcripts from 12 of 20 tested programs contained incorrect information, including recording a different drug than the one the doctor actually prescribed, alongside hallucinated content and privacy risks. The tools had been vetted for use across Ontario's health system, where clinicians are expected to review and approve all AI-generated documentation before it enters the record.

Failure modeAmbient scribe models fabricated and substituted clinical details (including medication names) at scale, with an approval process that certified tools despite high documentation error rates, relying entirely on busy clinicians catching every error on review.

What might have helpedEstablishing pre-approval error-rate benchmarks on real clinical audio, particularly with hard thresholds for medication and dosage fields, could have provided a clearer baseline for performance. Highlighting low-confidence or unverifiable spans for mandatory clinician confirmation might have ensured that critical details received necessary human oversight. Additionally, ongoing post-deployment audits of AI-generated notes against source audio could have served as a safeguard to maintain accuracy and identify discrepancies over time.

What happenedPatient Jose Saucedo filed a proposed class action in San Diego Superior Court in late November 2025 against Sharp HealthCare and its affiliated medical groups. Sharp had started using Abridge, an ambient AI app that records patient-clinician conversations and drafts clinical notes, in April 2025. Saucedo says that after a July 2025 visit to a Sharp Rees-Stealy clinic he discovered from his own visit notes that the appointment had been recorded. The suit claims Abridge automatically inserted incorrect statements into charts saying patients 'were advised' the visit was recorded and had 'consented,' even when no such conversation took place. It also alleges that Sharp obtained no written authorization, offered no opt-out, and let vendor personnel access recordings and transcripts. According to the complaint, Sharp apologized to Saucedo and said the audio could stay on the vendor's servers for about a month. Plaintiffs estimate more than 100,000 Sharp patients may have been recorded. The suit relies on California's all-party consent, wiretapping and medical confidentiality laws, and asks for damages, correction of medical records and an injunction. Sharp declined to comment on pending litigation. These are allegations that have not been tested in court.

Failure modeThe AI scribe allegedly generated boilerplate consent statements that were never verified and placed them in the medical record. That falsely documented informed consent for every visit it recorded and hid that consent was missing. The notes were apparently signed off without clinicians checking or correcting those statements.

What might have helpedCoverage of the case suggests several things that could have helped. Patients could have been asked for consent, and offered a way to decline, before recording began; KPBS reported that UC San Diego Health and Kaiser require consent. Clinicians could have checked statements of fact such as consent before signing AI-drafted notes, rather than letting template text become part of the record. Keeping consent as a step the clinician performs, and not text the AI generates, could have stopped the false attestations.

What happenedAP investigation found OpenAI's Whisper — used by medical transcription vendors in clinics — fabricates sentences, including invented medications and racial commentary, sometimes deleting the source audio after transcription.

Failure modeHallucination in silence/gaps; deployment in clinical documentation despite explicit vendor warnings against high-risk use; audio deletion removed the verification path.

What might have helpedRetaining source audio and applying confidence flagging to low-signal segments could have provided a basis for verification. Having clinicians review transcripts before chart entry might have caught errors that automated systems missed. One safeguard would be to avoid deploying general-purpose automatic speech recognition in clinical pipelines without specific domain validation, as this approach could have reduced the risk of misinterpretation.

What happenedClinicians and regulators documented Babylon's symptom-checker triaging classic heart-attack presentations (central chest pain in a 60-year-old smoker) to non-urgent pathways, among other unsafe outputs surfaced by Twitter user @DrMurphy11 and BBC reporting.

Failure modeSymptom-checker under-triage of time-critical conditions; marketing claims ('on par with doctors') outran validation; slow response to external safety reports.

What might have helpedTreating red-flag symptom lists as triggers for automatic escalation could have ensured consistent responses to critical signs. Implementing adversarial clinical testing prior to launch might have identified potential safety gaps earlier. Establishing a dedicated vulnerability-report channel for clinicians could have provided a structured way to surface concerns and improve system resilience.