Defining and benchmarking open problems in single-cell analysis.
Malte Daniel LueckenScott A GiganteDaniel BurkhardtRobrecht CannoodtDaniel StroblNikolay MarkovLuke ZappiaGiovanni PallaWesley LewisDaniel DimitrovMichael VinyardDaniel MagruderAlma AnderssonEmma DannQian QinDominik J OttoMichal KleinOlga BotvinnikLouise DeconinckKai WaldrantBastian RieckConstantin Ahlmann-EltzeEduardo da Veiga BeltrameAndrew BenzCarmen Bravo González-BlasAnn ChenBenjamin DeMeoCan ErgenSwann Floc'hlayAdam GayosoStephanie C HicksYuge JiVitalii KleshchevnikovGioele La MannoMaximilian LombardoRomain LopezDario RighelliHirak SarkarValentine SvenssonAlexander TongGalen XingChenling XuJonathan BloomAngela PiscoJulio Saez-RodriguezDrausin WulsinLuca PinelloYvan SaeysFabian Joachim TheisSmita KrishnaswamyPublished in: Research square (2024)
With the growing number of single-cell analysis tools, benchmarks are increasingly important to guide analysis and method development. However, a lack of standardisation and extensibility in current benchmarks limits their usability, longevity, and relevance to the community. We present Open Problems, a living, extensible, community-guided benchmarking platform including 10 current single-cell tasks that we envision will raise standards for the selection, evaluation, and development of methods in single-cell analysis.