DeepSTEP -- Deep Learning-Based Spatio-Temporal End-To-End Perception for Autonomous Vehicles

• DOI: 10.48550/arxiv.2305.06820
• Publication Date: 2023-01-01
• Authors: Sebastian Huch; Florian Sauerbeck; Johannes Betz
• Keywords: End-to-end principle; Robustness; Sensor Fusion

Autonomous vehicles demand high accuracy and robustness of perception algorithms. To develop efficient scalable algorithms, the maximum information should be extracted from available sensor data. In this work, we present our concept for an end-to-end architecture, named DeepSTEP. The deep learning-based architecture processes raw data camera, LiDAR, RaDAR, combines in a fusion network. output network is shared feature space, which used by head networks to fulfill several tasks, such as object detection or local mapping. DeepSTEP incorporates multiple ideas advance state art: First, combining localization into single pipeline allows processing reduce computational overhead further improves overall performance. Second, leverages temporal domain using self-attention mechanism that focuses on most important features. We believe will development systems. deployed research vehicle, platform collection, real-world testing, validation. conclusion, represents significant advancement field autonomous vehicles. architecture's design, time-aware attention mechanism, integration tasks make it promising solution deployment. This work progress presents first establishing novel pipeline.