Loss-Balanced Task Weighting to Reduce Negative Transfer in Multi-Task Learning

• DOI: 10.1609/aaai.v33i01.33019977
• Publication Date: 2019-08-15T07:37:35Z
• Authors: Shengchao Liu; Yingyu Liang; Anthony Gitter
• Keywords: 0301 basic medicine; 03 medical and health sciences

In settings with related prediction tasks, integrated multi-task learning models can often improve performance relative to independent single-task models. However, even when the average task performance improves, individual tasks may experience negative transfer in which the multi-task model’s predictions are worse than the single-task model’s. We show the prevalence of negative transfer in a computational chemistry case study with 128 tasks and introduce a framework that provides a foundation for reducing negative transfer in multitask models. Our Loss-Balanced Task Weighting approach dynamically updates task weights during model training to control the influence of individual tasks.