Why do users override alerts? Utilizing large language model to summarize comments and optimize clinical decision support.
Siru LiuAllison B McCoyAileen P WrightScott D NelsonSean S HuangHasan B AhmadSabrina E CarroJacob FranklinJames BroganAdam WrightPublished in: Journal of the American Medical Informatics Association : JAMIA (2024)
End-user comments provide clinicians' immediate feedback to CDS alerts and can serve as a direct and valuable data resource for improving CDS delivery. Traditionally, these comments may not be considered in the CDS review process due to their unstructured nature, large volume, and the presence of redundant or irrelevant content. Our study demonstrates that GPT-4 is capable of distilling these comments into summaries characterized by high clarity, accuracy, and completeness. AI-generated summaries are equivalent and potentially better than human-generated summaries. These AI-generated summaries could provide CDS experts with a novel means of reviewing user comments to rapidly optimize CDS alerts both online and offline.