Reproducible variability: assessing investigator discordance across 9 research teams attempting to reproduce the same observational study.
Anna OstropoletsYasser AlbogamiMitchell ConoverJuan M BandaWilliam A BaumgartnerClair BlacketerPriyamvada DesaiScott L DuVallStephen FortinJames P GilbertAsieh GolozarJoshua IdeAndrew S KanterDavid M KernChungsoo KimLana Y H LaiChenyu LiFeifan LiuKristine E LynchEvan MintyMaria Inês NevesDing Quan NgTontel ObeneVictor PeraNicole PrattGowtham RaoNadav RappoportInes ReineckePaola SaroufimAzza ShoaibiKatherine SimonMarc A SuchardJoel N SwerdelErica A VossJames WeaverLinying ZhangGeorge HripcsakPatrick B RyanPublished in: Journal of the American Medical Informatics Association : JAMIA (2023)
Independent research teams attempting to reproduce the study based on its free-text description alone produce different implementations that vary in the population size and composition. Sharing analytical code supported by a common data model and open-source tools allows reproducing a study unambiguously thereby preserving initial design choices.