COVID-19 pneumonia chest radiographic severity score: variability assessment among experienced and in-training radiologists and creation of a multireader composite score database for artificial intelligence algorithm development.
Marly van AssenMohammadreza ZandehshahvarHossein MalekiYashar KiarashiTimothy ArleoArthur E StillmanPeter FilevAmir H DavarpanahEugene A BerkowitzStefan TiggesScott J LeeBrianna L VeyAli AdibiCarlo N De CeccoPublished in: The British journal of radiology (2022)
Most AI algorithms are trained on data labeled by a single expert. This study shows that for COVID-19 X-ray severity classification there is significant variability and disagreement between radiologist and between residents.