Human-Level Play in the Game of Diplomacy by Combining Language Models with Strategic Reasoning

• DOI: 10.5281/zenodo.7236700
• Publication Date: 2022-11-22
• Authors: Anton Bakhtin; Noam Brown; Emily Dinan; Gabriele Farina; Colin Flaherty; Daniel Fried; Andrew Goff; Jonathan Gray; Hengyuan Hu; Athul Paul Jacob; Mojtaba Komeili; Karthik Konath; Adam Lerer; Michael Lewis; Alexander Miller; Sasha Mitts; Adithya Renduchin...; Stephen Roller; Dirk Rowe; Weiyan Shi; Joe Spisak; Alexander Wei; David Wu; Hugh Zhang; Markus Zijlstra; Minae Kwon
• Keywords: 0301 basic medicine; 03 medical and health sciences; Video Games; Artificial Intelligence; Humans; Software; Diplomacy; Language

Despite much progress in training artificial intelligence (AI) systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce Cicero, the first AI agent to achieve human-level performance inDiplomacy, a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. Cicero integrates a language model with planning and reinforcement learning algorithms by inferring players’ beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous onlineDiplomacyleague, Cicero achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.