Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

• DOI: 10.1016/j.eng.2023.03.018
• Publication Date: 2023-06-23T00:39:34Z
• Authors: Kang Yuan; Yanjun Huang; Shuo Yang; Zewei Zhou; Yulei Wang; Dongpu Cao; Hong Chen
• Keywords: Deep reinforcement learning; 0502 economics and business; 05 social sciences; Autonomous driving; Motion planning; Model predictive control; TA1-2040; Engineering (General). Civil engineering (General); Decision-making

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment. This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data- and model-driven method. First, a data-driven decision-making module based on deep reinforcement learning (DRL) is developed to pursue a rational driving performance as much as possible. Then, model predictive control (MPC) is employed to execute both longitudinal and lateral motion planning tasks. Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements. Finally, two principles of safety and rationality for the self-evolution of autonomous driving are proposed. A motion envelope is established and embedded into a rational exploration and exploitation scheme, which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent. Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted, and the results show that the proposed online-evolution framework is able to generate safer, more rational, and more efficient driving action in a real-world environment.