Providentia - A Large-Scale Sensor System for the Assistance of Autonomous Vehicles and Its Evaluation

• DOI: 10.48550/arxiv.1906.06789
• Publication Date: 2022-06-01
• Authors: Annkathrin Krammer; Christoph Schöller; Dhiraj Gulati; Venkatnarayanan L...; Franz Kurz; Dominik Rosenbaum; Claus Lenz; Alois Knoll
• Keywords: FOS: Computer and information sciences; data fusion; —Intelligent infrastructure system; sensor system; Computer Vision and Pattern Recognition (cs.CV); 05 social sciences; Computer Science - Computer Vision and Pattern Recognition; extended environmental perception; Computer Science - Robotics; evaluation with aerial images; autonomous driving; digital twin; 0502 economics and business; Robotics (cs.RO)

The environmental perception of an autonomous vehicle is limited by its physical sensor ranges and algorithmic performance, as well as by occlusions that degrade its understanding of an ongoing traffic situation. This not only poses a significant threat to safety and limits driving speeds, but it can also lead to inconvenient maneuvers. Intelligent Infrastructure Systems can help to alleviate these problems. An Intelligent Infrastructure System can fill in the gaps in a vehicle's perception and extend its field of view by providing additional detailed information about its surroundings, in the form of a digital model of the current traffic situation, i.e. a digital twin. However, detailed descriptions of such systems and working prototypes demonstrating their feasibility are scarce. In this paper, we propose a hardware and software architecture that enables such a reliable Intelligent Infrastructure System to be built. We have implemented this system in the real world and demonstrate its ability to create an accurate digital twin of an extended highway stretch, thus enhancing an autonomous vehicle's perception beyond the limits of its on-board sensors. Furthermore, we evaluate the accuracy and reliability of the digital twin by using aerial images and earth observation methods for generating ground truth data.<br/>Accepted for publication in the Journal of Field Robotics<br/>