Improving Model-Based Control and Active Exploration with Reconstruction Uncertainty Optimization

• DOI: 10.48550/arxiv.1812.03955
• Publication Date: 2018-01-01
• Authors: Di Palo, Norman; Valpola, Harri
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; 0209 industrial biotechnology; Computer Science - Artificial Intelligence; Computer Science - Neural and Evolutionary Computing; Machine Learning (stat.ML); 02 engineering and technology; Machine Learning (cs.LG); Computer Science - Robotics; Artificial Intelligence (cs.AI); Statistics - Machine Learning; 0202 electrical engineering, electronic engineering, information engineering; Neural and Evolutionary Computing (cs.NE); Robotics (cs.RO)

Model based predictions of future trajectories of a dynamical system often suffer from inaccuracies, forcing model based control algorithms to re-plan often, thus being computationally expensive, suboptimal and not reliable. In this work, we propose a model agnostic method for estimating the uncertainty of a model?s predictions based on reconstruction error, using it in control and exploration. As our experiments show, this uncertainty estimation can be used to improve control performance on a wide variety of environments by choosing predictions of which the model is confident. It can also be used for active learning to explore more efficiently the environment by planning for trajectories with high uncertainty, allowing faster model learning.