Unity in Diversity: Collaborative Pre-training Across Multimodal Medical Sources

• DOI: 10.18653/v1/2024.acl-long.199
• Publication Date: 2024-09-20T19:33:08Z
• Authors: Xiaochen Wang; Junyu Luo; Jiaqi Wang; Yuan Zhong; Xiaokun Zhang; Yaqing Wang; Parminder Bhatia; Cao Xiao; Fenglong Ma
• Keywords: Unity in diversity

Although pre-training has become a prevalent approach for addressing various biomedical tasks, the current efficacy of pre-trained models is hindered by their reliance on limited scope medical sources. This limitation results in data scarcity during and restricts range applicable downstream tasks. In response to these challenges, we develop Medical Cross-Source Pre-training (MEDCSP), new strategy designed bridge gap between multimodal MEDCSP employs modality-level aggregation unify patient within individual Additionally, leveraging temporal information diagnosis history, effectively captures explicit implicit correlations patients across different To evaluate proposed strategy, conduct comprehensive experiments, where experiments are based 6 modalities from 2 real-world sources, evaluated 4 tasks against 19 baselines, marking an initial yet essential step towards cross-source modeling domain.