Automated Utterance Labeling of Conversations Using Natural Language Processing

• DOI: 10.48550/arxiv.2208.06525
• Publication Date: 2022-01-01
• Authors: Maria Laricheva; Chiyu Zhang; Yan Liu; Guanyu Chen; Terence Tracey; Richard Young; Giuseppe Carenini
• Keywords: FOS: Computer and information sciences; Computer Science - Computation and Language; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Computation and Language (cs.CL)

Conversational data is essential in psychology because it can help researchers understand individuals cognitive processes, emotions, and behaviors. Utterance labelling is a common strategy for analyzing this type of data. The development of NLP algorithms allows researchers to automate this task. However, psychological conversational data present some challenges to NLP researchers, including multilabel classification, a large number of classes, and limited available data. This study explored how automated labels generated by NLP methods are comparable to human labels in the context of conversations on adulthood transition. We proposed strategies to handle three common challenges raised in psychological studies. Our findings showed that the deep learning method with domain adaptation (RoBERTa-CON) outperformed all other machine learning methods; and the hierarchical labelling system that we proposed was shown to help researchers strategically analyze conversational data. Our Python code and NLP model are available at https://github.com/mlaricheva/automated_labeling.<br/>Accepted in SBP-BRiMS 2022 (Camera-ready version)<br/>