CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues

• DOI: 10.48550/arxiv.2404.03820
• Publication Date: 2024-04-04
• Authors: Makesh Narsimhan ...; Traian Rebedea; Shaona Ghosh; Christopher Parisien
• Keywords: FOS: Computer and information sciences; Computer Science - Computation and Language; Computation and Language (cs.CL)

Recent advancements in instruction-tuning datasets have predominantly focused on specific tasks like mathematical or logical reasoning. There has been a notable gap data designed for aligning language models to maintain topic relevance conversations - critical aspect deploying chatbots production. We introduce the CantTalkAboutThis dataset help remain subject at hand during task-oriented interactions. It consists of synthetic dialogues wide range conversation topics from different domains. These are interspersed with distractor turns that intentionally divert chatbot predefined topic. Fine-tuning this helps make them resilient deviating role assigned and improves their ability topical coherence compared general-purpose instruction-tuned LLMs GPT-4-turbo Mixtral-Instruct. Additionally, preliminary observations suggest training also enhance performance fine-grained instruction following tasks.