Katherine E. Brown; Jesse O. Wrenn; Nicholas J. Jackson; Michael R. Cauley; Benjamin X. Collins; Laurie L. Novak; Bradley A. Malin; Jessica S. Ancker (2026).听.听Journal of the American Medical Informatics Association, 33(3), 621鈥�631.听
This study examines how machine learning (ML) systems鈥攐ften used to support healthcare decisions鈥攃an sometimes produce unfair results, meaning their predictions may be less accurate for certain patient groups. A key concern is that clinicians may rely too heavily on these systems, which can unintentionally reinforce these biases. The researchers explored a strategy called听ML suppression, which means selectively 鈥渟ilencing鈥� or withholding certain AI predictions when they are likely to be unreliable, based on an auditing process. They also looked at whether incorporating听uncertainty estimates听(how confident the model is in its predictions) could help decide when to suppress outputs.
Using large hospital datasets, the team simulated how clinicians and ML systems would work together to predict outcomes like death, ICU admission, or hospital readmission. They compared different scenarios, including when the AI performed better than clinicians and when it performed worse. They evaluated both accuracy (using a standard metric called AUC, which measures how well predictions distinguish outcomes) and fairness (measured by differences in error rates across groups).
The results showed that when the AI model performed better than clinicians, using suppression improved overall performance without making fairness worse. When clinicians performed better, relying on human judgment alone was often as fair or fairer than using suppressed AI predictions. Importantly, adding uncertainty information helped improve results further by better identifying when AI predictions should be ignored. Overall, the study suggests that carefully filtering out low-quality AI predictions can improve both the effectiveness and fairness of human鈥揂I collaboration in healthcare.
听
Figure 1.
Schematic indicating the collaboration scenario with and without suppression.34