Evaluating the performance of artificial intelligence-based software for digital mammography characterization

• DOI: 10.17816/dd625967
• Publication Date: 2024-12-02T14:42:46Z
• Authors: Yuriy A. Vasilev; Alexander V. Kols...; Kirill M. Arzamasov; Anton V. Vladzymy...; Olga V. Omelyanskaya; Serafim S. Semenov; Lubov E. Axenova
• Keywords: digital mammography; Computer applications to medicine. Medical informatics; R858-859.7; diagnostic accuracy; youden’s index; malignant tumors of breast; artificial intelligence services; 0101 mathematics; 01 natural sciences

BACKGROUND: Digital screening mammography is a key modality for early detection of breast cancer, reducing mortality by 20–40%. Many artificial intelligence (AI)-based services have been developed to automate the analysis of imaging data. AIM: The aim of the study was to compare mammography assessments using three types of AI services in multiple versions with radiologists’ conclusions. MATERIALS AND METHODS: Binary mammography scoring scales were compared with several types and versions of AI services regarding diagnostic accuracy, Matthews correlation coefficient, and maximum Youden’s index. RESULTS: A comparative analysis showed that the use of a binary scale for evaluating digital mammography affects the number of detected abnormalities and accuracy of AI results. In addition, diagnostic accuracy was found to be threshold dependent. AI Service 1 in version 3 had the best performance, as confirmed by most diagnostic accuracy parameters. CONCLUSIONS: Our results can be used to select AI services for interpreting mammography screening data. Using Youden’s index maximization to set up an AI service provides a balance of sensitivity and specificity that is not always clinically relevant.