Deep features-based speech emotion recognition for smart affective services

• DOI: 10.1007/s11042-017-5292-7
• Publication Date: 2017-10-31T06:17:11Z
• Authors: Abdul Malik Badshah; Nasir Rahim; Noor Ullah; Jamil Ahmad; Khan Muhammad; Mi Young Lee; Soonil Kwon; Sung Wook Baik
• Keywords: Spectrogram; Discriminative model; Pooling

Emotion recognition from speech signals is an interesting research with several applications like smart healthcare, autonomous voice response systems, assessing situational seriousness by caller affective state analysis in emergency centers, and other smart affective services. In this paper, we present a study of speech emotion recognition based on the features extracted from spectrograms using a deep convolutional neural network (CNN) with rectangular kernels. Typically, CNNs have square shaped kernels and pooling operators at various layers, which are suited for 2D image data. However, in case of spectrograms, the information is encoded in a slightly different manner. Time is represented along the x-axis and y-axis shows frequency of the speech signal, whereas, the amplitude is indicated by the intensity value in the spectrogram at a particular position. To analyze speech through spectrograms, we propose rectangular kernels of varying shapes and sizes, along with max pooling in rectangular neighborhoods, to extract discriminative features. The proposed scheme effectively learns discriminative features from speech spectrograms and performs better than many state-of-the-art techniques when evaluated its performance on Emo-DB and Korean speech dataset.