Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition

• DOI: 10.21437/interspeech.2020-1221
• Publication Date: 2020-10-27T09:22:11Z
• Authors: Chao Weng; Chengzhu Yu; Jia Cui; Chunlei Zhang; Dong Yu
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Sound (cs.SD); Computer Science - Computation and Language; 02 engineering and technology; Computer Science - Sound; Machine Learning (cs.LG); 03 medical and health sciences; Audio and Speech Processing (eess.AS); 0202 electrical engineering, electronic engineering, information engineering; FOS: Electrical engineering, electronic engineering, information engineering; 0305 other medical science; Computation and Language (cs.CL); Electrical Engineering and Systems Science - Audio and Speech Processing

In this work, we propose minimum Bayes risk (MBR) training of RNN-Transducer (RNN-T) for end-to-end speech recognition. Specifically, initialized with a RNN-T trained model, MBR training is conducted via minimizing the expected edit distance between the reference label sequence and on-the-fly generated N-best hypothesis. We also introduce a heuristic to incorporate an external neural network language model (NNLM) in RNN-T beam search decoding and explore MBR training with the external NNLM. Experimental results demonstrate an MBR trained model outperforms a RNN-T trained model substantially and further improvements can be achieved if trained with an external NNLM. Our best MBR trained system achieves absolute character error rate (CER) reductions of 1.2% and 0.5% on read and spontaneous Mandarin speech respectively over a strong convolution and transformer based RNN-T baseline trained on ~21,000 hours of speech.