USimAgent: Large Language Models for Simulating Search Users

• DOI: 10.48550/arxiv.2403.09142
• Publication Date: 2024-03-14
• Authors: Erhan Zhang; X. Wang; Peiyuan Gong; Yankai Lin; Jiaxin Mao
• Keywords: FOS: Computer and information sciences; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Information Retrieval (cs.IR); Computer Science - Information Retrieval

Due to the advantages in cost-efficiency and reproducibility, user simulation has become a promising solution user-centric evaluation of information retrieval systems. Nonetheless, accurately simulating search behaviors long been challenge, because users' actions are highly complex driven by intricate cognitive processes such as learning, reasoning, planning. Recently, Large Language Models (LLMs) have demonstrated remarked potential human-level intelligence used building autonomous agents for various tasks. However, using LLMs not yet fully explored. In this paper, we introduce LLM-based behavior simulator, USimAgent. The proposed simulator can simulate querying, clicking, stopping during search, thus, is capable generating complete sessions specific Empirical investigation on real dataset shows that outperforms existing methods query generation comparable traditional predicting clicks behaviors. These results only validate effectiveness but also shed light development more robust generic simulators.