SVDinsTN: A Tensor Network Paradigm for Efficient Structure Search from Regularized Modeling Perspective

• DOI: 10.48550/arxiv.2305.14912
• Publication Date: 2023-01-01
• Authors: Yu‐Bang Zheng; Xi-Le Zhao; Junhua Zeng; Chao Li; Qibin Zhao; Heng-Chao Li; Ting‐Zhu Huang
• Keywords: Representation

Tensor network (TN) representation is a powerful technique for computer vision and machine learning. TN structure search (TN-SS) aims to customized achieve compact representation, which challenging NP-hard problem. Recent "sampling-evaluation"-based methods require sampling an extensive collection of structures evaluating them one by one, resulting in prohibitively high computational costs. To address this issue, we propose novel paradigm, named SVD-inspired decomposition (SVDinsTN), allows us efficiently solve the TN-SS problem from regularized modeling perspective, eliminating repeated evaluations. be specific, inserting diagonal factor each edge fully-connected TN, SVDinsTN calculate cores factors simultaneously, with sparsity revealing structure. In theory, prove convergence guarantee proposed method. Experimental results demonstrate that method achieves approximately 100 1000 times acceleration compared state-of-the-art while maintaining comparable level ability.