COVID-19 pneumonia chest radiographic severity score: variability assessment among experienced and in-training radiologists and creation of a multireader composite score database for artificial intelligence algorithm development

• DOI: 10.1259/bjr.20211028
• Publication Date: 2022-04-22T12:35:08Z
• Authors: Marly van Assen; Mohammadreza Zand...; Hossein Maleki; Yashar Kiarashi; Timothy Arleo; Arthur E. Stillman; Peter Filev; Amir H. Davarpanah; Eugene A. Berkowitz; Stefan Tigges; Scott J. Lee; Brianna L. Vey; Ali Adibi; Carlo N. De Cecco
• Keywords: Chest radiograph

Objective: The purpose was to evaluate reader variability between experienced and in-training radiologists of COVID-19 pneumonia severity on chest radiograph (CXR), create a multireader database suitable for AI development. Methods: In this study, CXRs from polymerase chain reaction positive patients were reviewed. Six cardiothoracic two residents classified each CXR according severity. One radiologist performed the classification twice assess intraobserver variability. Severity assessed using 4-class system: normal (0), mild (1), moderate (2), severe (3). A median score (Rad Med) determined six development (XCOMS). Kendal Tau correlation percentage disagreement calculated Results: total 397 (1208 CXRs) included (mean age, 60 years SD ± 1), 189 men). Interobserver ranges 0.67 0.78. Compared Rad Med score, show good 0.79–0.88. Residents slightly lower interobserver agreement 0.66 with other 0.69 0.71 radiologists. Intraobserver high coefficient 0.77. 220 (18%), 707 (59%), 259 (21%) 22 (2%) there 0, 1, 2 or 3 class-difference. 594 (50%) scores similar, in 578 (48%) 36 (3%) 1 Conclusion: Experienced demonstrate inter- classification. higher observed cases, which may affect training algorithms. Advances knowledge: Most algorithms are trained data labeled by single expert. This study shows that X-ray is significant residents.