Fusion is Not Enough: Single-Modal Attacks to Compromise Fusion Models in Autonomous Driving

• DOI: 10.48550/arxiv.2304.14614
• Publication Date: 2023-01-01
• Authors: Zhiyuan Cheng; Hongjun Choi; James Liang; Shiwei Feng; Guanhong Tao; Dongfang Liu; Michael Zuzak; Xiangyu Zhang
• Keywords: Modality (human–computer interaction); Sensor Fusion

Multi-sensor fusion (MSF) is widely used in autonomous vehicles (AVs) for perception, particularly 3D object detection with camera and LiDAR sensors. The purpose of to capitalize on the advantages each modality while minimizing its weaknesses. Advanced deep neural network (DNN)-based techniques have demonstrated exceptional industry-leading performance. Due redundant information multiple modalities, MSF also recognized as a general defence strategy against adversarial attacks. In this paper, we attack models from that considered be lesser importance but more affordable attackers. We argue weakest link depends their most vulnerable modality, propose an framework targets advanced camera-LiDAR fusion-based through camera-only Our approach employs two-stage optimization-based first thoroughly evaluates image areas under attacks, then applies dedicated strategies different generate deployable patches. evaluations six one model indicate our attacks successfully compromise all them. can either decrease mean average precision (mAP) performance 0.824 0.353, or degrade score target 0.728 0.156, demonstrating efficacy proposed framework. Code available.