PASSION: Towards Effective Incomplete Multi-Modal Medical Image Segmentation with Imbalanced Missing Rates

• DOI: 10.48550/arxiv.2407.14796
• Publication Date: 2024-07-20
• Authors: Junjie Shi; C. Shang; Zhaobin Sun; Li Yu; Xin Yang; Zengqiang Yan
• Keywords: Passion

Incomplete multi-modal image segmentation is a fundamental task in medical imaging to refine deployment efficiency when only partial modalities are available. However, the common practice that complete-modality data visible during model training far from realistic, as can have imbalanced missing rates clinical scenarios. In this paper, we, for first time, formulate such challenging setting and propose Preference-Aware Self-diStillatION (PASSION) incomplete under rates. Specifically, we construct pixel-wise semantic-wise self-distillation balance optimization objective of each modality. Then, define relative preference evaluate dominance modality training, based on which design task-wise gradient-wise regularization convergence different modalities. Experimental results two publicly available datasets demonstrate superiority PASSION against existing approaches balancing. More importantly, validated work plug-and-play module consistent performance improvement across backbones. Code at https://github.com/Jun-Jie-Shi/PASSION.