Debiasing Multimodal Sarcasm Detection with Contrastive Learning

• DOI: 10.1609/aaai.v38i16.29795
• Publication Date: 2024-03-25T11:46:58Z
• Authors: Mengzhao Jia; Can Xie; Liqiang Jing
• Keywords: Sarcasm; Debiasing

Despite commendable achievements made by existing work, prevailing multimodal sarcasm detection studies rely more on textual content over visual information. It unavoidably induces spurious correlations between words and labels, thereby significantly hindering the models' generalization capability. To address this problem, we define task of out-of-distribution (OOD) detection, which aims to evaluate generalizability when word distribution is different in training testing settings. Moreover, propose a novel debiasing framework with contrastive learning, mitigate harmful effect biased factors for robust OOD generalization. In particular, first design counterfactual data augmentation construct positive samples dissimilar biases negative similar biases. Subsequently, devise an adapted learning mechanism empower model learn task-relevant features alleviate adverse words. Extensive experiments show superiority proposed framework.