Cross-scale multi-instance learning for pathological image diagnosis

• DOI: 10.1016/j.media.2024.103124
• Publication Date: 2024-02-27T17:22:42Z
• Authors: Ruining Deng; Can Cui; Lucas W. Remedios; Shunxing Bao; R. Michael Womick; Sophie Chiron; Jia Li; Joseph T. Roland; Ken S. Lau; Qi Liu; Keith T. Wilson; Yaohong Wang; Lori A. Coburn; Bennett A. Landman; Yuankai Huo
• Keywords: 0301 basic medicine; FOS: Computer and information sciences; Computer Science - Machine Learning; 03 medical and health sciences; Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Computer Science - Computer Vision and Pattern Recognition; FOS: Electrical engineering, electronic engineering, information engineering; Humans; Electrical Engineering and Systems Science - Image and Video Processing; Algorithms; Machine Learning (cs.LG)

Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20x magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is three-fold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.