Deep Probabilistic Accelerated Evaluation: A Robust Certifiable Rare-Event Simulation Methodology for Black-Box Safety-Critical Systems

• DOI: 10.48550/arxiv.2006.15722
• Publication Date: 2020-01-01
• Authors: Mansur Arief; Zhiyuan Huang; Guru Koushik Sent...; Yuanlu Bai; Shengyi He; Wenhao Ding; Henry Lam; Ding Zhao
• Keywords: Black box; Deep Neural Networks; Certificate

Evaluating the reliability of intelligent physical systems against rare safety-critical events poses a huge testing burden for real-world applications. Simulation provides useful platform to evaluate extremal risks these before their deployments. Importance Sampling (IS), while proven be powerful rare-event simulation, faces challenges in handling learning-based due black-box nature that fundamentally undermines its efficiency guarantee, which can lead under-estimation without diagnostically detected. We propose framework called Deep Probabilistic Accelerated Evaluation (Deep-PrAE) design statistically guaranteed IS, by converting samplers are versatile but could lack guarantees, into one with what we call relaxed certificate allows accurate estimation bounds on event probability. present theory Deep-PrAE combines dominating point concept set learning via deep neural network classifiers, and demonstrate effectiveness numerical examples including safety-testing an driving algorithm.