The Energy and Carbon Footprint of Training End-to-End Speech Recognizers

• DOI: 10.21437/interspeech.2021-456
• Publication Date: 2021-08-27T05:59:39Z
• Authors: Titouan Parcollet; Mirco Ravanelli
• Keywords: Carbon Footprint; Footprint; End-to-end principle

Deep learning contributes to reaching higher levels of artificial intelligence. Due its pervasive adoption, however, growing concerns on the environmental impact this technology have been raised. In particular, energy consumed at training and inference time by modern neural networks is far from being negligible will increase even further due deployment ever larger models. This work investigates for first carbon cost end-to-end automatic speech recognition (ASR). First, it quantifies amount CO2 emitted while state-of-the-art (SOTA) ASR systems a university-scale cluster. Then, shows that tiny performance improvement comes an extremely high cost. For instance, conducted experiments reveal SOTA Transformer emits 50% total released solely achieve final decrease 0.3 word error rate. With study, we hope raise awareness crucial topic provide guidelines, insights, estimates enabling researchers better assess technologies.