Federated Pseudo Modality Generation for Incomplete Multi-Modal MRI Reconstruction

• DOI: 10.48550/arxiv.2308.10910
• Publication Date: 2023-01-01
• Authors: Yunlu Yan; Chun-Mei Feng; Yuexiang Li; Rick Siow Mong Goh; Lei Zhu
• Keywords: Modality (human–computer interaction); Centroid; Modalities; Code (set theory); Data Sharing

While multi-modal learning has been widely used for MRI reconstruction, it relies on paired data which is difficult to acquire in real clinical scenarios. Especially the federated setting, common situation that several medical institutions only have single-modal data, termed modality missing issue. Therefore, infeasible deploy a standard framework such conditions. In this paper, we propose novel communication-efficient framework, namely Fed-PMG, address challenge reconstruction. Specifically, utilize pseudo generation mechanism recover each client by sharing distribution information of amplitude spectrum frequency space. However, step original leads heavy communication costs. To reduce cost, introduce clustering scheme project set into finite cluster centroids, and share them among clients. With an elaborate design, our approach can effectively complete within acceptable cost. Extensive experiments demonstrate proposed method attain similar performance with ideal scenario, i.e., all clients full modalities. The source code will be released.