Revisiting Generalization Power of a DNN in Terms of Symbolic Interactions

• DOI: 10.48550/arxiv.2502.10162
• Publication Date: 2025-01-01
• Authors: Cheng, Lei; Zhang, Junpeng; Ren, Qihan; Zhang, Quanshi
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Science - Computation and Language; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; Computation and Language (cs.CL); Machine Learning (cs.LG)

This paper aims to analyze the generalization power of deep neural networks (DNNs) from the perspective of interactions. Unlike previous analysis of a DNN's generalization power in a highdimensional feature space, we find that the generalization power of a DNN can be explained as the generalization power of the interactions. We found that the generalizable interactions follow a decay-shaped distribution, while non-generalizable interactions follow a spindle-shaped distribution. Furthermore, our theory can effectively disentangle these two types of interactions from a DNN. We have verified that our theory can well match real interactions in a DNN in experiments.<br/>arXiv admin note: text overlap with arXiv:2407.19198<br/>