From Health 201AI Med Risk

documented ai-in-healthcare failures — what went wrong, why, and what might have helped, synthesised from public sources


Stanford Medicine's COVID-19 vaccine allocation formula prioritised only 7 of 1,300+ frontline residents

What happenedIn December 2020 Stanford Medicine used a rules-based formula to rank staff for its first 5,000 COVID-19 vaccine doses. The formula weighed factors such as age, job role and department. Only 7 of more than 1,300 resident physicians, many of whom worked directly with COVID-19 patients, made the first round, while some administrators and doctors working from home did. Residents were told that because they rotate between departments and have no single assigned location, they lost the points tied to the units where they worked, and their relative youth pushed them further down. On 18 December at least 100 residents protested. Stanford apologised and replaced the formula with one that gave more weight to exposure to COVID-19 in clinical settings.

Failure modeThe allocation formula relied on administrative data (assigned location, age) that didn't reflect actual clinical exposure, so the people at highest risk were ranked low.

What might have helpedChecking the ranked output against common sense, for example asking whether the staff who treat COVID-19 patients were near the top, could have caught the flaw before it was announced. Involving the affected groups, such as residents, in reviewing the formula could also have helped.

Fix adopted Since thenStanford Medicine apologised and dropped the formula. The new allocation ranked staff by clinical exposure: high-acuity settings (ED, ICU, COVID units) came first, then lower-risk hospital areas, then ambulatory and remote staff. Faculty who had received early doses waited until frontline residents and fellows were vaccinated. By 21 December more than 600 residents and fellows had been vaccinated, and on 22 December Stanford set up a public distribution dashboard.

Still openThe formula was tested and reviewed only after it had excluded residents and they protested. We found no published review of how the rules that went wrong were designed and approved.

LessonThe case suggests that a simple rules-based formula can go wrong in ways nobody sees until it runs on real staff lists. Testing it against the groups it is meant to protect before using it, and publishing results openly (as the dashboard later did), might have caught the problem sooner.

Sources: stanforddaily.com · npr.org

Sources: technologyreview.com · npr.org · statnews.com · sources re-checked 2026-09-26
← Back to the incident library

Not medical or legal advice. This site documents how AI systems have failed; it is not a source of care or advice. For a health problem, seek medical attention or talk to a health professional; in an emergency, call 911 or your local emergency number. For legal questions, consult a lawyer.

If you are in crisis: in the US, call or text 988 (Suicide & Crisis Lifeline). Elsewhere, see findahelpline.com.

To report harm from an AI tool, contact the relevant authority: see where to report a problem. AI Med Risk does not take complaints or act on individual cases.