MEATRD: Multimodal Anomalous Tissue Region Detection Enhanced with Spatial Transcriptomics

• DOI: 10.1609/aaai.v39i12.33409
• Publication Date: 2025-04-11T12:10:55Z
• Authors: Kaichen Xu; Qilong Wu; Yan Lu; Yinan Zheng; Wenlin Li; Xingjie Tang; Jun Wang; Xiaobo Sun

The detection of anomalous tissue regions (ATRs) within affected tissues is crucial in clinical diagnosis and pathological studies. Conventional automated ATR methods, primarily based on histology images alone, falter cases where ATRs normal have subtle visual differences. recent spatial transcriptomics (ST) technology profiles gene expressions across regions, offering a molecular perspective for detecting ATRs. However, there dearth methods that effectively harness complementary information from both ST. To address this gap, we propose MEATRD, novel method integrates image ST data. MEATRD trained to reconstruct patches expression spots (inliers) their multimodal embeddings, followed by learning one-class classification AD model latent reconstruction errors. This strategy harmonizes the strengths reconstruction-based approaches. At heart an innovative masked graph dual-attention transformer (MGDAT) network, which not only facilitates cross-modality cross-node sharing but also addresses over-generalization issue commonly seen methods. Additionally, demonstrate modality-specific, task-relevant collated condensed bottleneck encoding generated MGDAT, marking first theoretical analysis informational properties encoding. Extensive evaluations eight real datasets reveal MEATRD's superior performance detection, surpassing various state-of-the-art Remarkably, proves adept at discerning show slight deviations tissues.