Multimodal Recommender Systems: A Survey

• DOI: 10.48550/arxiv.2302.03883
• Publication Date: 2024-10-10
• Authors: Qidong Liu; Jiaxi Hu; Yutian Xiao; Xiangyu Zhao; Jingtong Gao; Wanyu Wang; Qing Li; Jiliang Tang
• Keywords: FOS: Computer and information sciences; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Information Retrieval (cs.IR); Computer Science - Information Retrieval

The recommender system (RS) has been an integral toolkit of online services. They are equipped with various deep learning techniques to model user preference based on identifier and attribute information. With the emergence of multimedia services, such as short videos, news, and and so on, understanding these contents while recommending becomes critical. Besides, multimodal features are also helpful in alleviating the problem of data sparsity in RS. Thus, M ultimodal R ecommender S ystem (MRS) has attracted much attention from both academia and industry recently. In this article, we will give a comprehensive survey of the MRS models, mainly from technical views. First, we conclude the general procedures and major challenges for MRS. Then, we introduce the existing MRS models according to four categories, i.e., Modality Encoder , Feature Interaction , Feature Enhancement , and Model Optimization . Besides, to make it convenient for those who want to research this field, we also summarize the dataset and code resources. Finally, we discuss some promising future directions of MRS and conclude this article. To access more details of the surveyed articles, such as implementation code, we open source a repository. 1