Evaluating Model-Free Reinforcement Learning toward Safety-Critical Tasks

• DOI: 10.1609/aaai.v37i12.26786
• Publication Date: 2023-06-27T18:22:26Z
• Authors: Linrui Zhang; Qin Zhang; Li Shen; Bo Yuan; Xueqian Wang; Dacheng Tao
• Keywords: Robustness

Safety comes first in many real-world applications involving autonomous agents. Despite a large number of reinforcement learning (RL) methods focusing on safety-critical tasks, there is still lack high-quality evaluation those algorithms that adheres to safety constraints at each decision step under complex and unknown dynamics. In this paper, we revisit prior work scope from the perspective state-wise safe RL categorize them as projection-based, recovery-based, optimization-based approaches, respectively. Furthermore, propose Unrolling Layer (USL), joint method combines optimization projection. This novel technique explicitly enforces hard via deep unrolling architecture enjoys structural advantages navigating trade-off between reward improvement constraint satisfaction. To facilitate further research area, reproduce related unified pipeline incorporate into SafeRL-Kit, toolkit provides off-the-shelf interfaces utilities for tasks. We then perform comparative study involved six benchmarks ranging robotic control driving. The empirical results provide an insight their applicability robustness zero-cost-return policies without task-dependent handcrafting. project page available https://sites.google.com/view/saferlkit.