SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages

• DOI: 10.48550/arxiv.2406.10118
• Publication Date: 2024-06-14
• Authors: Holy Lovenia; Rahmad Mahendra; Salsabil Maulana ...; Lester James V. M...; Jennifer Santoso; Elyanah Aco; Akhdan Fadhilah; Jonibek Mansurov; Joseph Marvin Imp...; Onno Kampman; Joel Ruben Antony...; Muhammad Ravi Shu...; Frederikus Hudi; Railey Montalan; Ryan Ignatius; Joanito Agili Lopo; William Nixon; Börje F. Karlsson; James Jaya; Ryandito Diandaru; Yuze Gao; Patrick Amadeus; B.-H Wang; Jan Christian Bla...; Chenxi Whitehouse; Ivan Halim Parmon...; Maria Khelli; Wenyu Zhang; Lucky Susanto; Reynard Adha Ryanda; Sonny Lazuardi He...; Dan John Velasco; Muhammad Dehan Al...; Willy Fitra Hendria; Yasmin Moslem; Noah R. Flynn; Muhammad Farid Ad...; Haochen Li; Johanes Lee; R. Damanhuri; Shuo Sun; Muhammad Reza Qorib; Amirbek Djanibekov; Wei Qi Leong; Quyet V. Do; Niklas Muennighoff; Tanrada Pansuwan; Ilham Firdausi Putra; Yan Xu; Ngee Chia Tai; Ayu Purwarianti; Sebastian Ruder; William Tjhi; Peerat Limkonchot...; Alham Fikri Aji; Sedrick Scott Keh; Genta Indra Winata; Ruochen Zhang; Fajri Koto; Zheng-Xin Yong; Samuel Cahyawijaya
• Keywords: Benchmark (surveying)

Southeast Asia (SEA) is a region rich in linguistic diversity and cultural variety, with over 1,300 indigenous languages population of 671 million people. However, prevailing AI models suffer from significant lack representation texts, images, audio datasets SEA, compromising the quality for SEA languages. Evaluating challenging due to scarcity high-quality datasets, compounded by dominance English training data, raising concerns about potential misrepresentation. To address these challenges, we introduce SEACrowd, collaborative initiative that consolidates comprehensive resource hub fills gap providing standardized corpora nearly 1,000 across three modalities. Through our SEACrowd benchmarks, assess on 36 13 tasks, offering valuable insights into current landscape SEA. Furthermore, propose strategies facilitate greater advancements, maximizing utility equity future