Audio Contrastive based Fine-tuning

• DOI: 10.48550/arxiv.2309.11895
• Publication Date: 2023-01-01
• Authors: Wang, Yang; Liang, Qibin; Xiao, Chenghao; Li, Yizhi; Moubayed, Noura Al; Lin, Chenghua
• Keywords: FOS: Computer and information sciences; Sound (cs.SD); Computer Science - Computation and Language; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Audio and Speech Processing (eess.AS); FOS: Electrical engineering, electronic engineering, information engineering; Computation and Language (cs.CL); Computer Science - Sound; Electrical Engineering and Systems Science - Audio and Speech Processing

Audio classification plays a crucial role in speech and sound processing tasks with a wide range of applications. There still remains a challenge of striking the right balance between fitting the model to the training data (avoiding overfitting) and enabling it to generalise well to a new domain. Leveraging the transferability of contrastive learning, we introduce Audio Contrastive-based Fine-tuning (AudioConFit), an efficient approach characterised by robust generalisability. Empirical experiments on a variety of audio classification tasks demonstrate the effectiveness and robustness of our approach, which achieves state-of-the-art results in various settings.<br/>Under review<br/>