Monolingual versus Multilingual BERTology for Vietnamese Extractive Multi-Document Summarization

• DOI: 10.48550/arxiv.2108.13741
• Publication Date: 2021-01-01
• Authors: To, Huy Quoc; Van Nguyen, Kiet; Nguyen, Ngan Luu-...; Nguyen, Anh Gia-Tuan
• Keywords: FOS: Computer and information sciences; Computer Science - Computation and Language; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; 01 natural sciences; Computation and Language (cs.CL); 0105 earth and related environmental sciences

Recent researches have demonstrated that BERT shows potential in a wide range of natural language processing tasks. It is adopted as an encoder for many state-of-the-art automatic summarizing systems, which achieve excellent performance. However, so far, there is not much work done for Vietnamese. In this paper, we showcase how BERT can be implemented for extractive text summarization in Vietnamese on multi-document. We introduce a novel comparison between different multilingual and monolingual BERT models. The experiment results indicate that monolingual models produce promising results compared to other multilingual models and previous text summarizing models for Vietnamese.