Predictability and Surprise in Large Generative Models

• DOI: 10.1145/3531146.3533229
• Publication Date: 2022-06-20T14:27:10Z
• Authors: Deep Ganguli; Danny Hernandez; Liane Lovitt; Amanda Askell; Yuntao Bai; Anna Chen; Tom Conerly; Nova Dassarma; Dawn Drain; Nelson Elhage; Sheer El Showk; Stanislav Fort; Zac Hatfield-Dodds; Tom Henighan; Scott Johnston; Andy Jones; Nicholas Joseph; Jackson Kernian; Shauna Kravec; Ben Mann; Neel Nanda; Kamal Ndousse; Catherine Olsson; Daniela Amodei; Tom Brown; Jared Kaplan; Sam McCandlish; Christopher Olah; Dario Amodei; Jack Clark
• Keywords: FOS: Computer and information sciences; Computer Science - Computers and Society; Computers and Society (cs.CY); 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; 01 natural sciences; 0105 earth and related environmental sciences

Updated to reflect the version submitted (and accepted) to ACM FAccT '22. This update incorporates feedback from peer-review and fixes minor typos. See open access FAccT conference version at: https://dl.acm.org/doi/abs/10.1145/3531146.3533229<br/>Large-scale pre-training has recently emerged as a technique for creating capable, general purpose, generative models such as GPT-3, Megatron-Turing NLG, Gopher, and many others. In this paper, we highlight a counterintuitive property of such models and discuss the policy implications of this property. Namely, these generative models have an unusual combination of predictable loss on a broad training distribution (as embodied in their "scaling laws"), and unpredictable specific capabilities, inputs, and outputs. We believe that the high-level predictability and appearance of useful capabilities drives rapid development of such models, while the unpredictable qualities make it difficult to anticipate the consequences of model deployment. We go through examples of how this combination can lead to socially harmful behavior with examples from the literature and real world observations, and we also perform two novel experiments to illustrate our point about harms from unpredictability. Furthermore, we analyze how these conflicting properties combine to give model developers various motivations for deploying these models, and challenges that can hinder deployment. We conclude with a list of possible interventions the AI community may take to increase the chance of these models having a beneficial impact. We intend this paper to be useful to policymakers who want to understand and regulate AI systems, technologists who care about the potential policy impact of their work, and academics who want to analyze, critique, and potentially develop large generative models.<br/>