Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Prediction

• DOI: 10.48550/arxiv.2303.10552
• Publication Date: 2023-01-01
• Authors: Haibao Yu; Yingjuan Tang; Enze Xie; Jilei Mao; Jirui Yuan; Ping Luo; Zaiqing Nie
• Keywords: Asynchrony (computer programming); Feature (linguistics); Code (set theory)

Cooperatively utilizing both ego-vehicle and infrastructure sensor data can significantly enhance autonomous driving perception abilities. However, temporal asynchrony limited wireless communication in traffic environments lead to fusion misalignment impact detection performance. This paper proposes Feature Flow Net (FFNet), a novel cooperative framework that uses feature flow prediction module address these issues vehicle-infrastructure 3D object detection. Rather than transmitting maps extracted from still-images, FFNet transmits flow, which leverages the coherence of sequential frames predict future features compensate for asynchrony. Additionally, we introduce self-supervised approach enable generate with ability. Experimental results demonstrate our proposed method outperforms existing methods while requiring no more 1/10 transmission cost raw on DAIR-V2X dataset when exceeds 200$ms$. The code is available at \href{https://github.com/haibao-yu/FFNet-VIC3D}{https://github.com/haibao-yu/FFNet-VIC3D}.