Evaluating the robustness of deep learning models trained to diagnose idiopathic pulmonary fibrosis using a retrospective study

• DOI: 10.1002/mp.17752
• Publication Date: 2025-03-23T05:38:28Z
• Authors: Wenxi Yu; Michael F. McNitt...; Jonathan G Goldin; Jin Woo Song; Grace Hyun J. Kim
• Keywords: Supine position; Robustness

Deep learning (DL)-based systems have not yet been broadly implemented in clinical practice, part due to unknown robustness across multiple imaging protocols. To this end, we aim evaluate the performance of several previously developed DL-based models, which were trained distinguish idiopathic pulmonary fibrosis (IPF) from non-IPF among interstitial lung disease (ILD) patients, under standardized reference CT In study, utilized scans ILD subjects, acquired using various protocols, assess model performance. Three including one 2D and two 3D classify patients into IPF or based on chest scans. These models image data 389 700 retrospectively, obtained five multicenter studies. For some (e.g., at inhalation exhalation) and/or reconstructed thin slice thick slice). Thus, for each patient, dataset was selected be used construction classification model, so parameters that set serve as conditions. its specific study protocol, many had sets both prone supine positions different parameters. Therefore, non-reference) identified 343 subjects who condition (used construction) non-reference conditions evaluation conditions), analysis. We reported specificities three Generalized linear mixed effects (GLMM) identify significant technical associated with getting inconsistent diagnostic results between Selected include effective tube current-time product (known "effective mAs"), reconstruction kernels, thickness, patient orientation (prone supine), scanner diagnosis. Limitations retrospective nature study. all DL overall specificity diagnosis decreased (p < 0.05 out models). GLMM further suggests least mean mAs scan is key factor leads decrease predictive 0.001); difference = 0.03) thickness (3 mm; p are flagged factors models; other statistically > 0.05). Preliminary findings demonstrated lack when applied series collected indicated care should taken acquisition developing deploying practice.