Towards Red Teaming in Multimodal and Multilingual Translation

• DOI: 10.48550/arxiv.2401.16247
• Publication Date: 2024-01-29
• Authors: Christophe Ropers; David C. Dale; Prangthip Hansanti; Gabriel Mejia Gon...; Ivan Evtimov; C. I. Wong; Christophe Touret; Kristina Pereyra; Seohyun Sonia Kim; Cristian Canton F...; Pierre Andrews; Marta R. Costa‐jussà

Assessing performance in Natural Language Processing is becoming increasingly complex. One particular challenge the potential for evaluation datasets to overlap with training data, either directly or indirectly, which can lead skewed results and overestimation of model performance. As a consequence, human gaining increasing interest as means assess reliability models. such method red teaming approach, aims generate edge cases where will produce critical errors. While this methodology standard practice generative AI, its application realm conditional AI remains largely unexplored. This paper presents first study on human-based Machine Translation (MT), marking significant step towards understanding improving translation We delve into both automation, reporting lessons learned providing recommendations models drills. pioneering work opens up new avenues research development field MT.