Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation

• DOI: 10.18653/v1/2021.emnlp-main.511
• Publication Date: 2021-12-16T22:56:42Z
• Authors: Yingjie Li; Chenye Zhao; Cornelia Caragea
• Keywords: Margin (machine learning); Code (set theory); Scarcity

Stance detection determines whether the author of a text is in favor of, against or neutral to specific target and provides valuable insights into important events such as legalization abortion. Despite significant progress on this task, one remaining challenges scarcity annotations. Besides, most previous works focused hard-label training which meaningful similarities among categories are discarded during training. To address these challenges, first, we evaluate multi-target multi-dataset settings by model each dataset datasets different domains, respectively. We show that models can learn more universal representations with respect targets settings. Second, investigate knowledge distillation stance observe transferring from teacher student be beneficial our proposed Moreover, propose an Adaptive Knowledge Distillation (AKD) method applies instance-specific temperature scaling predictions. Results performs best all it further improved AKD, outperforming state-of-the-art large margin. publicly release code.