How does the artificial intelligence-based image-assisted technique help physicians in diagnosis of pulmonary adenocarcinoma? A randomized controlled experiment of multicenter physicians in China

• DOI: 10.1093/jamia/ocac179
• Publication Date: 2022-10-13T19:06:55Z
• Authors: Jiaoyang Li; Lingxiao Zhou; Yi Zhan; Haifeng Xu; Cheng Zhang; Fei Shan; Lei Liu
• Keywords: Lung Neoplasms; Artificial Intelligence; Radiologists; Humans; Adenocarcinoma of Lung; Sensitivity and Specificity; 3. Good health

Abstract

Objective Although artificial intelligence (AI) has achieved high levels of accuracy in the diagnosis of various diseases, its impact on physicians’ decision-making performance in clinical practice is uncertain. This study aims to assess the impact of AI on the diagnostic performance of physicians with differing levels of self-efficacy under working conditions involving different time pressures.

Materials and methods A 2 (independent diagnosis vs AI-assisted diagnosis) × 2 (no time pressure vs 2-minute time limit) randomized controlled experiment of multicenter physicians was conducted. Participants diagnosed 10 pulmonary adenocarcinoma cases and their diagnostic accuracy, sensitivity, and specificity were evaluated. Data analysis was performed using multilevel logistic regression.

Results One hundred and four radiologists from 102 hospitals completed the experiment. The results reveal (1) AI greatly increases physicians’ diagnostic accuracy, either with or without time pressure; (2) when no time pressure, AI significantly improves physicians’ diagnostic sensitivity but no significant change in specificity, while under time pressure, physicians’ diagnostic sensitivity and specificity are both improved with the aid of AI; (3) when no time pressure, physicians with low self-efficacy benefit from AI assistance thus improving diagnostic accuracy but those with high self-efficacy do not, whereas physicians with low and high levels of self-efficacy both benefit from AI under time pressure.

Discussion This study is one of the first to provide real-world evidence regarding the impact of AI on physicians’ decision-making performance, taking into account 2 boundary factors: clinical time pressure and physicians’ self-efficacy.

Conclusion AI-assisted diagnosis should be prioritized for physicians working under time pressure or with low self-efficacy.