Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning

• DOI: 10.48550/arxiv.2107.12544
• Publication Date: 2021-01-01
• Authors: Pedro A. Tsividis; João Loula; Jake Burga; Nathan Foss; Andres Campero; Thomas Pouncy; Samuel J. Gershman; Joshua B. Tenenbaum
• Keywords: Video game; Generative model

Reinforcement learning (RL) studies how an agent comes to achieve reward in environment through interactions over time. Recent advances machine RL have surpassed human expertise at the world's oldest board games and many classic video games, but they require vast quantities of experience learn successfully -- none today's algorithms account for ability so different tasks, quickly. Here we propose a new approach this challenge based on particularly strong form model-based which call Theory-Based Learning, because it uses human-like intuitive theories rich, abstract, causal models physical objects, intentional agents, their explore model environment, plan effectively task goals. We instantiate game playing called EMPA (the Exploring, Modeling, Planning Agent), performs Bayesian inference probabilistic generative expressed as programs game-engine simulator, runs internal simulations these support efficient object-based, relational exploration heuristic planning. closely matches efficiency suite 90 challenging Atari-style just minutes play generalizing robustly situations levels. The also captures fine-grained structure people's trajectories dynamics. Its design behavior suggest way forward building more general AI systems.