Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States.
Estee Y CramerEvan L RayVelma K LopezJohannes BracherAndrea BrennenAlvaro J Castro RivadeneiraAaron GerdingTilmann GneitingKatie H HouseYuxin HuangDasuni JayawardenaAbdul H KanjiAyush KhandelwalKhoa LeAnja MühlemannJarad NiemiApurv ShahAriane StarkYijin WangNutcha WattanachitMartha W ZornYouyang GuSansiddh JainNayana BannurAyush DevaMihir KulkarniSrujana MeruguAlpan RavalSiddhant ShingiAvtansh TiwariJerome WhiteNeil F AbernethySpencer WoodyMaytal DahanSpencer J FoxKelly GaitherMichael LachmannLauren Ancel MeyersJames G ScottMauricio TecAjitesh SrivastavaGlover E GeorgeJeffrey C CeganIan D DettwillerWilliam P EnglandMatthew W FarthingRobert H HunterBrandon LaffertyIgor LinkovMichael L MayoMatthew D ParnoMichael A RowlandBenjamin D TrumpYanli Zhang-JamesSamuel ChenStephen V FaraoneJonathan HessChristopher P MorleyAsif SalekinDongliang WangSabrina M CorsettiThomas M BaerMarisa C EisenbergKarl FalbYitao HuangEmily T MartinElla McCauleyRobert L MyersTom SchwarzDaniel R SheldonGraham Casey GibsonRose YuLiyao GaoYian MaDongxia WuXifeng YanXiaoyong JinYu-Xiang WangYangQuan ChenLihong GuoYanting ZhaoQuanquan GuJinghui ChenLingxiao WangPan XuWeitong ZhangDifan ZouHannah BiegelJoceline LegaSteve McConnellV P NagrajStephanie L GuertinChristopher Hulme-LoweStephen D TurnerYunfeng ShiXuegang BanRobert WalravenQi-Jun HongStanley KongAxel van de WalleJames A TurtleMichal Ben-NunSteven RileyPete RileyUgur KoyluogluDavid DesRochesPedro ForliBruce HamoryChristina KyriakidesHelen LeisJohn MillikenMichael MoloneyJames MorganNinad NirgudkarGokce OzcanNoah PiwonkaMatt RaviChris SchraderElizabeth ShakhnovichDaniel SiegelRyan SpatzChris StiefelingBarrie WilkinsonAlexander WongSean M CavanyGuido EspañaSean M MooreRachel J OidtmanT Alex PerkinsDavid KrausAndrea KrausZhifeng GaoJiang BianWei CaoJuan M Lavista FerresChaozhuo LiTie-Yan LiuXing XieShun ZhangShun ZhengAlessandro VespignaniMatteo ChinazziJessica T DavisKunpeng MuAna Pastore Y PionttiXinyue XiongAndrew ZhengJackie BaekVivek FariasAndreea GeorgescuRetsef LeviDeeksha SinhaJoshua WildeGeorgia PerakisMohammed Amine BennounaDavid Nze-NdongDivya SinghviIoannis SpantidakisLeann ThayaparanAsterios TsiourvasArnab SarkerAli JadbabaieDevavrat ShahNicolas Della PennaLeo Anthony CeliSaketh SundarRuss WolfingerDave OsthusLauren CastroGeoffrey FairchildIsaac MichaudDean KarlenMatt KinseyLuke C MullanyKaitlin Rainwater-LovettLauren ShinKatharine TallaksenShelby WilsonElizabeth C LeeJuan Dent HulseKyra H GrantzAlison L HillJoshua KaminskyKathryn KaminskyLindsay T KeeganStephen A LauerJoseph Chadi LemaitreJustin LesslerHannah R MeredithJavier Perez-SaezSam ShahClaire P SmithShaun A TrueloveJosh WillsMaximilian MarshallLauren GardnerKristen NixonJohn C BurantLily WangLei GaoZhiling GuMyungjin KimXinyi LiGuannan WangYueying WangShan YuRobert C ReinerRyan BarberEmmanuela GakidouSimon I HaySteve LimChristopher J L MurrayDavid PigottHeidi L GurungPrasith BaccamSteven A StageBradley T SuchoskiB Aditya PrakashBijaya AdhikariJiaming CuiAlexander RodríguezAnika TabassumJiajia XiePınar KeskinocakJohn AsplundArden BaxterBuse Eylul OrucNicoleta SerbanSercan O ArikMike DusenberryArkady EpshteynElli KanalLong T LeChun-Liang LiTomas PfisterDario SavaRajarishi SinhaThomas TsaiNathanael C YoderJinsung YoonLeyou ZhangSam AbbottNikos I BosseSebastian FunkJoel HellewellSophie R MeakinKatharine SherratMingyuan ZhouRahi KalantariTeresa K YamanaSen PeiJeffrey L ShamanMichael Lingzhi LiDimitris BertsimasOmar Skali LamiSaksham SoniHamza Tazi BouardiTurgay AyerMadeline AdeeJagpreet ChhatwalOzden O DalgicMary A LaddBenjamin P LinasPeter MuellerJade XiaoYuanjia WangQinxia WangShanghong XieDonglin ZengAlden GreenJacob BienLogan BrooksAddison J HuMaria JahjaDaniel J McDonaldBalasubramanian NarasimhanCollin A PolitschSamyak RajanalaAaron RumackNoah SimonRyan J TibshiraniRob TibshiraniValerie VenturaLarry WassermanEamon B O'DeaJohn M DrakeRobert PaganoQuoc T TranLam Si Tung HoHuong HuynhJo W WalkerRachel B SlaytonMichael A JohanssonMatthew BiggerstaffNicholas G ReichPublished in: Proceedings of the National Academy of Sciences of the United States of America (2022)
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks.
Keyphrases
- coronavirus disease
- sars cov
- healthcare
- public health
- mental health
- respiratory syndrome coronavirus
- cardiovascular disease
- emergency department
- quality improvement
- cardiovascular events
- machine learning
- randomized controlled trial
- coronary artery disease
- risk factors
- convolutional neural network
- health information
- health insurance