Quantifying the redundancy between prosody and text

• DOI: 10.18653/v1/2023.emnlp-main.606
• Publication Date: 2023-12-10T16:58:19Z
• Authors: Lukas Wolf; Tiago Pimentel; Evelina Fedorenko; Ryan Cotterell; Alex Warstadt; Ethan Wilcox; Tamar Regev
• Keywords: Pitch accent

Prosody—the suprasegmental component of speech, including pitch, loudness, and tempo—carries critical aspects meaning. However, the relationship between information conveyed by prosody vs. words themselves remains poorly understood. We use large language models (LLMs) to estimate how much is redundant themselves. Using a spoken corpus English audiobooks, we extract prosodic features aligned individual test well they can be predicted from LLM embeddings, compared non-contextual word embeddings. find high degree redundancy carried across several features, intensity, duration, pauses, pitch contours. Furthermore, word’s with both itself context preceding as following it. Still, observe that not fully text, suggesting carries above beyond words. Along this paper, release general-purpose data processing pipeline for quantifying linguistic extra-linguistic features.