LangXAI: Integrating Large Vision Models for Generating Textual Explanations to Enhance Explainability in Visual Perception Tasks

• DOI: 10.24963/ijcai.2024/1025
• Publication Date: 2024-07-26T14:28:11Z
• Authors: Hung Nguyen; Tobias Clement; Loc Nguyen; Nils Kemmerzell; Binh Truong; Khang Nguyen; Mohamed Abdelaal; Hung Cao
• Keywords: Computer Science - Artificial Intelligence; Computer Science - Computer Vision and Pattern Recognition

LangXAI is a framework that integrates Explainable Artificial Intelligence (XAI) with advanced vision models to generate textual explanations for visual recognition tasks. Despite XAI advancements, an understanding gap persists for end-users with limited domain knowledge in artificial intelligence and computer vision. LangXAI addresses this by furnishing text-based explanations for classification, object detection, and semantic segmentation model outputs to end-users. Preliminary results demonstrate LangXAI's enhanced plausibility, with high BERTScore across tasks, fostering a more transparent and reliable AI framework on vision tasks for end-users. The code and demo of this work can be found at https://analytics-everywhere-lab.github.io/langxai.io/.