Unveiling Mode Connectivity in Graph Neural Networks

• DOI: 10.48550/arxiv.2502.12608
• Publication Date: 2025-02-18
• Authors: Bingheng Li; Zhikai Chen; Haoyu Han; Shenglai Zeng; Jingzhe Liu; Jiliang Tang
• Keywords: Mode (computer interface)

A fundamental challenge in understanding graph neural networks (GNNs) lies characterizing their optimization dynamics and loss landscape geometry, critical for improving interpretability robustness. While mode connectivity, a lens analyzing geometric properties of landscapes has proven insightful other deep learning architectures, its implications GNNs remain unexplored. This work presents the first investigation connectivity GNNs. We uncover that exhibit distinct non-linear diverging from patterns observed fully-connected or CNNs. Crucially, we demonstrate structure, rather than model architecture, dominates this behavior, with like homophily correlating patterns. further establish link between generalization, proposing generalization bound based on barriers revealing utility as diagnostic tool. Our findings bridge theoretical insights practical implications: they rationalize domain alignment strategies provide foundation refining GNN training paradigms.