Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model.
Eliane RöösliSelen BozkurtTina Hernandez BoussardPublished in: Scientific data (2022)
As artificial intelligence (AI) makes continuous progress to improve quality of care for some patients by leveraging ever increasing amounts of digital health data, others are left behind. Empirical evaluation studies are required to keep biased AI models from reinforcing systemic health disparities faced by minority populations through dangerous feedback loops. The aim of this study is to raise broad awareness of the pervasive challenges around bias and fairness in risk prediction models. We performed a case study on a MIMIC-trained benchmarking model using a broadly applicable fairness and generalizability assessment framework. While open-science benchmarks are crucial to overcome many study limitations today, this case study revealed a strong class imbalance problem as well as fairness concerns for Black and publicly insured ICU patients. Therefore, we advocate for the widespread use of comprehensive fairness and performance assessment frameworks to effectively monitor and validate benchmark pipelines built on open data resources.
Keyphrases
- artificial intelligence
- healthcare
- big data
- public health
- ejection fraction
- end stage renal disease
- newly diagnosed
- machine learning
- prognostic factors
- minimally invasive
- mental health
- intensive care unit
- quality improvement
- climate change
- patient reported outcomes
- body composition
- binding protein
- single cell
- human health
- resistance training
- pain management
- health promotion