DEPHN: Different Expression Parallel Heterogeneous Network using virtual gradient optimization for Multi-task Learning

• DOI: 10.48550/arxiv.2307.12519
• Publication Date: 2023-01-01
• Authors: Menglin Kong; Ri Su; Shaojie Zhao; Muzhou Hou
• Keywords: Feature (linguistics); Representation; Expression (computer science)

Recommendation system algorithm based on multi-task learning (MTL) is the major method for Internet operators to understand users and predict their behaviors in multi-behavior scenario of platform. Task correlation an important consideration MTL goals, traditional models use shared-bottom gating experts realize shared representation information differentiation. However, The relationship between real-world tasks often more complex than existing methods do not handle properly sharing information. In this paper, we propose Different Expression Parallel Heterogeneous Network (DEPHN) model multiple simultaneously. DEPHN constructs at bottom by using different feature interaction improve generalization ability flow. view model's differentiating task flows, uses explicit mapping virtual gradient coefficient expert during training process, adaptively adjusts intensity gated unit considering difference values correlation. Extensive experiments artificial datasets demonstrate that our proposed can capture situations achieve better performance baseline models\footnote{Accepted IJCNN2023}.