Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation

• DOI: 10.48550/arxiv.2404.05970
• Publication Date: 2024-04-08
• Authors: Alireza Salemi; Surya Kallumadi; Hamed Zamani
• Keywords: FOS: Computer and information sciences; Computer Science - Computation and Language; Computation and Language (cs.CL); Information Retrieval (cs.IR); Computer Science - Information Retrieval

This paper studies retrieval-augmented approaches for personalizing large language models (LLMs), which potentially have a substantial impact on various applications and domains. We propose the first attempt to optimize retrieval that deliver limited number of personal documents purpose personalized generation. develop two optimization algorithms solicit feedback from downstream generation tasks optimization--one based reinforcement learning whose reward function is defined using any arbitrary metric another knowledge distillation LLM model. also introduces pre- post-generation retriever selection model decides what choose each input. Extensive experiments diverse personalization (LaMP) benchmark reveal statistically significant improvements in six out seven datasets.