The Impact of Longitudinal Data-Completeness of Electronic Health Record Data on the Prediction Performance of Clinical Risk Scores.

The impact of electronic health record (EHR) discontinuity, i.e., receiving care outside of a given EHR system, on EHR-based risk prediction is unknown. We aimed to assess the impact of EHR-continuity on performance of clinical risk scores. The study cohort consisted of patients aged ≥65 years with ≥1 EHR encounter in the two networks in Massachusetts (MA, 2007/1/1-2017/12/31, internal training and validation dataset), and one network in North Carolina (NC, 2007/1/1-2016/12/31, external validation dataset) that were linked with Medicare claims data. Risk scores were calculated using EHR data alone vs. linked EHR-claims data (not subject to misclassification due to EHR-discontinuity): 1) combined co-morbidity score (CCS), 2) claim-based frailty score (CFI), 3) CHAD 2 DS 2 -VASc, 4) HAS-BLED. We assessed the performance of CCS and CFI predicting death, CHAD 2 DS 2 -VASc predicting ischemic stroke, and HAS-BLED predicting bleeding by Area under ROC Curve (AUC), stratified by quartiles of predicted EHR-continuity (Q1-4). There were 319,740 patients in the MA systems and 125,380 in the NC system. In the external validation dataset, AUC for EHR-based CCS predicting one-year risk of death was 0.583 in Q1 (lowest) EHR-continuity group, which increased to 0.739 in Q4 (highest) EHR-continuity group. The corresponding improvement in AUC was 0.539 to 0.647 for CFI, 0.556 to 0.637 for CHAD 2 DS 2 -VASc, and 0.517 to 0.556 for HAS-BLED. The AUC in Q4 EHR-continuity group based on EHR alone approximates that based on EHR-claims data. The prediction performance of 4 clinical risk scores was substantially worse in patients with lower vs. high EHR-continuity.