Science study: widely used Optum care-management algorithm underestimated Black patients' health needs by using cost as a proxy
What happenedIn October 2019 Ziad Obermeyer and colleagues published a study in Science of a commercial algorithm, reported to be Optum's, that US health systems used to identify patients for high-risk care-management programmes. The algorithm predicted future healthcare costs as a stand-in for health need. Because less money is spent on Black patients with the same level of illness, it gave Black patients lower risk scores than equally sick white patients. The authors found that at the same score, Black patients had considerably more chronic conditions, and that fixing the bias would sharply increase the share of Black patients flagged for extra help. Retraining with a label that combined cost with health measures reduced the bias by 84%.
Failure modeLabel-choice bias: the model predicted healthcare spending instead of illness, building unequal access to care into who was flagged for extra help.
What might have helpedThe authors suggest that checking whether the prediction target actually measures health need, and auditing calibration across racial groups with direct health measures, could have revealed the gap. Their retraining with health-based labels showed the bias could be largely removed.
Partly mitigated Since thenHHS's 2024 Section 1557 rule (45 CFR 92.210) requires hospitals, clinics, physician groups and payers that receive federal funding to make reasonable efforts to find patient care decision-support tools, including algorithms and AI, that use factors measuring race and other protected traits, and to reduce the risk of discrimination from them. The compliance deadline was 1 May 2025.
Still openIt asks only for 'reasonable efforts', and it targets tools whose inputs measure a protected trait. A tool like this one, which used cost as a stand-in for need, may not be caught. Enforcement by HHS's Office for Civil Rights is uncertain after its 2025–2026 policy changes and reorganisation. We found no public statement from Optum on retraining the product.
LessonThe case suggests that bias can enter through what a model is trained to predict, even when race is not an input at all. Rules that check only the input variables may miss that kind of bias; the study's authors found it by comparing outcomes for Black and white patients.
Sources: federalregister.gov · healthaffairs.org · nature.com