MatterChat: A Multi-Modal LLM for Material Science

• DOI: 10.48550/arxiv.2502.13107
• Publication Date: 2025-02-18
• Authors: Yingheng Tang; Wenbin Xu; Jie Cao; Jianzhu Ma; Weilu Gao; Steve Farrell; Benjamin Erichson; Michael W. Mahoney; Andy Nonaka; Zhi Yao
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Machine Learning (cs.LG)

Understanding and predicting the properties of inorganic materials is crucial for accelerating advancements in science driving applications energy, electronics, beyond. Integrating material structure data with language-based information through multi-modal large language models (LLMs) offers great potential to support these efforts by enhancing human-AI interaction. However, a key challenge lies integrating atomic structures at full resolution into LLMs. In this work, we introduce MatterChat, versatile structure-aware LLM that unifies structural textual inputs single cohesive model. MatterChat employs bridging module effectively align pretrained machine learning interatomic LLM, reducing training costs flexibility. Our results demonstrate significantly improves performance property prediction interaction, surpassing general-purpose LLMs such as GPT-4. We also its usefulness more advanced scientific reasoning step-by-step synthesis.