Self-supervised speech models have grown fast during the past few years and proven feasible for use in various downstream tasks. Some recent work has started to look at characteristics of these models, yet many concerns not been fully addressed. In this work, we conduct a study on emotional corpora explore popular self-supervised model - wav2vec 2.0. Via set quantitative analysis, mainly demonstrate that: 1) 2.0 appears discard paralinguistic information that is less useful word recognition...

Recognizing emotional reactions of movie audiences to affective content is a challenging task in computing. Previous research on induced emotion recognition has mainly focused using audio-visual content. Nevertheless, the relationship between perceptions (perceived emotions) and emotions evoked (induced unexplored. In this work, we studied perceived audiences. Moreover, investigated multimodal modelling approaches predict from based features, as well physiological behavioral To carry out...

Automatic emotion recognition is vital for building natural and engaging human-computer interaction systems. Combining information from multiple modalities typically improves performance. In previous work, features different have generally been fused at the same level with two types of fusion strategies: Feature-Level fusion, which concatenates feature sets before recognition; Decision-Level makes final decision based on outputs unimodal models. However, may describe data time scales or...

Alongside acoustic information, linguistic features based on speech transcripts have been proven useful in Speech Emotion Recognition (SER). However, due to the scarcity of emotion labelled data and difficulty recognizing emotional speech, it is hard obtain reliable models this research area. In paper, we propose fuse Automatic (ASR) outputs into pipeline for joint training SER. The relationship between ASR SER understudied, unclear what how benefit By examining various fusion methods, our...

In this work, we compare emotion recognition on two types of speech: spontaneous and acted dialogues. Experiments were conducted the AVEC2012 database dialogues IEMOCAP We studied performance acoustic features for recognition: knowledge-inspired disfluency nonverbal vocalisation (DIS-NV) features, statistical Low-Level Descriptor (LLD) based features. Both Support Vector Machines (SVM) Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) built using each feature set emotional...

Predicting the emotional response of movie audiences to affective content is a challenging task in computing. Previous work has focused on using audiovisual predict induced emotions. However, relationship between audience's perceptions (perceived emotions) and emotions evoked audience (induced remains unexplored. In this work, we address perceived movies, identify features modelling approaches effective for predicting First, extend LIRIS-ACCEDE database by annotating crowd-sourced manner,...

Current multimodal sentiment analysis frames score prediction as a general Machine Learning task. However, what the actually represents has often been overlooked. As measurement of opinions and affective states, generally consists two aspects: polarity intensity. We decompose scores into these aspects study how they are conveyed through individual modalities combined models in naturalistic monologue setting. In particular, we build unimodal multi-task learning with main task and/or intensity...

Abstract The Lothian Diary Project is an interdisciplinary effort to collect self-recorded audio or video diaries of people’s experiences COVID-19 in and around Edinburgh, Scotland. In this paper we describe how the project emerged from a desire support community members. have been disseminated through public events, website, oral history project, engagement with policymakers. data collection method encouraged participation people disabilities, racialized individuals, immigrants,...

This paper investigates how rising intonation affects the interpretation of cue words in dialogue. Both and express a range speaker attitudes like uncertainty surprise. However, it is unclear perception these relates to dialogue structure belief co-ordination. Perception experiment results suggest that rises reflect difficulty integrating new information rather than signaling lack credibility. leads general analysis as current question under discussion unresolved. interaction with word...

In this study we present a performance comparison for five pitch extraction algorithms: auto correlation (AC), cross (CC), and sub-harmonic summation (as implemented in PRAAT [Boersma Weenick (2010)]), the robust algorithm tracking (RAPT) ESPS [Talkin (1995)], SWIPE’ [Camacho (2007)]. Recent research showed that SHS outperformed other algorithms on two speech databases with EGG reference values That study, however, used fixed search range of 40–800 Hz all speakers, regardless sex or...

In this paper we investigate how participant involvement and turn-taking features relate to extractive summarization of meeting dialogues. particular, examine whether automatically derived measures group level involvement, like participation equality freedom, can help detect where relevant segments will be. Results show that classification using performed better than the majority class baseline for data from both AMI ICSI corpora in identifying contain summary dialogue acts. The feature...

Speech synthesis has improved in both expressiveness and voice quality recent years.However, obtaining full when dealing with large multi-sentential synthesized discourse is still a challenge, since speech synthesizers do not take into account the prosodic differences that have been observed units such as paragraphs.The current study validates extends previous work by analyzing prosody of paragraph diverse corpus TED Talks using automatically extracted F0, intensity timing features.In...

We address the task of automatically predicting group satisfaction in meetings using acoustic, lexical, and turn-taking features. Participant is measured post-meeting ratings from AMI corpus. focus on three aspects satisfaction: overall satisfaction, participant attention information overload. All predictions are made at aggregated level. In general, we find that combining features across modalities improves prediction performance. However, feature ablation significantly Our experiments also...

This work aims to explore the correlation between discourse structure of a spoken monologue and its prosody by predicting relations from different prosodic attributes.For this purpose, corpus semi-spontaneous monologues in English has been automatically annotated according Rhetorical Structure Theory, which models coherence text via rhetorical relations.From corresponding audio files, features such as pitch, intensity, speech rate have extracted contexts relation.Supervised classification...

Being able to detect topics and speaker stances in conversations is a key requirement for developing spoken language understanding systems that are personalized adaptive.In this work, we explore how topic-oriented stance expressed conversational speech.To do this, present new set of topic annotations the CallHome corpus spontaneous dialogues.Specifically, focus on six stances-positivity, certainty, surprise, amusement, interest, comfort-which useful characterizing important aspects...

In Speech Emotion Recognition (SER), textual data is often used alongside audio signals to address their inherent variability.However, the reliance on human annotated text in most research hinders development of practical SER systems.To overcome this challenge, we investigate how Automatic (ASR) performs emotional speech by analyzing ASR performance emotion corpora and examining distribution word errors confidence scores transcripts gain insight into affects ASR.We utilize four systems,...