Cross-Task Knowledge Distillation in Multi-Task Recommendation

• DOI: 10.1609/aaai.v36i4.20352
• Publication Date: 2022-07-04T11:11:20Z
• Authors: Chenxiao Yang; Junwei Pan; Xiaofeng Gao; Tingyu Jiang; Dapeng Liu; Guihai Chen
• Keywords: Leverage (statistics)

Multi-task learning (MTL) has been widely used in recommender systems, wherein predicting each type of user feedback on items (e.g, click, purchase) are treated as individual tasks and jointly trained with a unified model. Our key observation is that the prediction results task may contain task-specific knowledge about user’s fine-grained preference towards items. While such could be transferred to benefit other tasks, it being overlooked under current MTL paradigm. This paper, instead, proposes Cross-Task Knowledge Distillation framework attempts leverage one supervised signals teach another task. However, integrating KD proper manner non-trivial due several challenges including conflicts, inconsistent magnitude requirement synchronous optimization. As countermeasures, we 1) introduce auxiliary quadruplet loss functions capture cross-task ranking information avoid 2) design calibrated distillation approach align distill from 3) propose novel error correction mechanism enable facilitate training teacher student models. Comprehensive experiments conducted verify effectiveness our real-world datasets.