Deep learning from multiple experts improves identification of amyloid neuropathologies

• DOI: 10.1186/s40478-022-01365-0
• Publication Date: 2022-04-28T15:04:25Z
• Authors: Daniel R. Wong; Ziqi Tang; Nicholas C. Mew; Sakshi Das; Justin Athey; Kirsty E. McAleese; Julia K. Kofler; Margaret E. Flanagan; Ewa Borys; Charles L. White; Atul J. Butte; Brittany N. Dugger; Michael J. Keiser
• Keywords: Identification

Pathologists can label pathologies differently, making it challenging to yield consistent assessments in the absence of one ground truth. To address this problem, we present a deep learning (DL) approach that draws on cohort experts, weighs each contribution, and is robust noisy labels. We collected 100,495 annotations 20,099 candidate amyloid beta neuropathologies (cerebral angiopathy (CAA), cored diffuse plaques) from three institutions, independently annotated by five experts. DL methods trained consensus-of-two strategy yielded 12.6-26% improvements area under precision recall curve (AUPRC) when compared those learned individualized annotations. This surpassed individual-expert models, even unfairly assessed benchmarks favoring them. Moreover, ensembling over individual models was hidden random annotators. In blind prospective tests 52,555 subsequent expert-annotated images, labeled like their human counterparts (consensus model AUPRC = 0.74 cored; 0.69 CAA). study demonstrates means combine multiple truths into common-ground yields diagnoses informed potentially variable expert opinions.