Do Text-to-Text Multi-Task Learners Suffer from Task Conflict?

• DOI: 10.48550/arxiv.2212.06645
• Publication Date: 2022-01-01
• Authors: David Mueller; Nicholas Andrews; Mark Dredze
• Keywords: Transfer of learning

Traditional multi-task learning architectures train a single model across multiple tasks through shared encoder followed by task-specific decoders. Learning these models often requires specialized training algorithms that address task-conflict in the parameter updates, which otherwise can lead to negative transfer. A new type of within NLP homogenizes as and language decoder, does surprisingly well range diverse tasks. Does this architecture suffer from task-conflicts require algorithms? We study how certain factors shift towards text-to-text affects conflict transfer, finding both directional transfer are constant architectures.