Comparison of Diagnostic and Triage Accuracy of Ada Health and WebMD Symptom Checkers, ChatGPT, and Physicians for Patients in an Emergency Department: Clinical Data Analysis Study.

Hamish S F Fraser Daven Crossland Ian Bacher Megan E Ranney Tracy E Madsen Ross Hilliard

Published in: JMIR mHealth and uHealth (2023)

ChatGPT 3.5 had high diagnostic accuracy but a high unsafe triage rate. ChatGPT 4.0 had the poorest diagnostic accuracy, but a lower unsafe triage rate and the highest triage agreement with the physicians. The Ada and WebMD SCs performed better overall than ChatGPT. Unsupervised patient use of ChatGPT for diagnosis and triage is not recommended without improvements to triage accuracy and extensive clinical evaluation.

Keyphrases

emergency department
data analysis
clinical evaluation
primary care
end stage renal disease
public health
healthcare
machine learning
ejection fraction
chronic kidney disease
prognostic factors
case report
health information
adverse drug
risk assessment
patient reported outcomes