Alignment Studio: Aligning Large Language Models to Particular Contextual Regulations

• DOI: 10.48550/arxiv.2403.09704
• Publication Date: 2024-03-08
• Authors: Swapnaja Achintalwar; Ioana Baldini; Djallel Bouneffouf; Joan Byamugisha; Maria Chang; Pierre Dognin; Eitan Farchi; Ndivhuwo Makondo; Aleksandra Mojsil...; Manish Nagireddy; Karthikeyan Nates...; Inkit Padhi; Orna Raz; Jesus Rios; Prasanna Sattigeri; Moninder Singh; Siphiwe Thwala; Rosario Uceda‐Sosa; Kush R. Varshney
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Science - Computation and Language; Artificial Intelligence (cs.AI); Computer Science - Artificial Intelligence; Computation and Language (cs.CL); Machine Learning (cs.LG)

The alignment of large language models is usually done by model providers to add or control behaviors that are common universally understood across use cases and contexts. In contrast, in this article, we present an approach architecture empowers application developers tune a their particular values, social norms, laws other regulations, orchestrate between potentially conflicting requirements context. We lay out three main components such Alignment Studio architecture: Framers, Instructors, Auditors work concert the behavior model. illustrate with running example aligning company's internal-facing enterprise chatbot its business conduct guidelines.