Adaptation of Whisper models to child speech recognition

• DOI: 10.21437/interspeech.2023-935
• Publication Date: 2023-08-14T08:22:20Z
• Authors: Rishabh Jain; Andrei Barcovschi; Mariam Yiwere; Peter Corcoran; Horia Cucu
• Keywords: FOS: Computer and information sciences; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Audio and Speech Processing (eess.AS); FOS: Electrical engineering, electronic engineering, information engineering; Electrical Engineering and Systems Science - Audio and Speech Processing

Automatic Speech Recognition (ASR) systems often struggle with transcribing child speech due to the lack of large child speech datasets required to accurately train child-friendly ASR models. However, there are huge amounts of annotated adult speech datasets which were used to create multilingual ASR models, such as Whisper. Our work aims to explore whether such models can be adapted to child speech to improve ASR for children. In addition, we compare Whisper child-adaptations with finetuned self-supervised models, such as wav2vec2. We demonstrate that finetuning Whisper on child speech yields significant improvements in ASR performance on child speech, compared to non finetuned Whisper models. Additionally, utilizing self-supervised Wav2vec2 models that have been finetuned on child speech outperforms Whisper finetuning.<br/>Accepted in Interspeech 2023<br/>