InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning

• DOI: 10.48550/arxiv.2502.11573
• Publication Date: 2025-02-17
• Authors: Chenglong Xie; Shengguan Cai; Wenjun Wang; Pengxiang Li; Zhijie Sang; Kejing Yang; Yiming Zhang; Zhen Li; Guanghao Zhu; Zeyu Liu; Yang Yu; Yuhang Liu; Lu Su; Baoyi He; Qi Zhou; Xiaotian Han; Jianbo Yuan; Shengyu Zhang; Fei Wu; Hongxia Yang
• Keywords: FOS: Computer and information sciences; Computer Science - Computation and Language; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Computation and Language (cs.CL)

Large Language Models (LLMs) and Multimodal (MLLMs) have made significant advancements in reasoning capabilities. However, they still face challenges such as high computational demands privacy concerns. This paper focuses on developing efficient Small (SLMs) (MSLMs) that retain competitive abilities. We introduce a novel training pipeline enhances capabilities facilitates deployment edge devices, achieving state-of-the-art performance while minimizing development costs. \InfR~ aims to advance AI systems by improving reasoning, reducing adoption barriers, addressing concerns through smaller model sizes. Resources are available at https://github. com/Reallm-Labs/InfiR.