TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation

• DOI: 10.48550/arxiv.2303.15771
• Publication Date: 2023-01-01
• Authors: Xiangyun Meng; Nathan O. Hatch; Alexander Lambert; Anqi Li; Nolan Wagener; Matthew Schmittle; JoonHo Lee; Wentao Yuan; Zoey Chen; Samuel Deng; Greg Okopal; Dieter Fox; Byron Boots; Amirreza Shaban
• Keywords: Feature (linguistics)

Effective use of camera-based vision systems is essential for robust performance in autonomous off-road driving, particularly the high-speed regime. Despite success structured, on-road settings, current end-to-end approaches scene prediction have yet to be successfully adapted complex outdoor terrain. To this end, we present TerrainNet, a vision-based terrain perception system semantic and geometric aggressive, navigation. The approach relies on several key insights practical considerations achieving reliable modeling. network includes multi-headed output representation capture fine- coarse-grained features necessary estimating traversability. Accurate depth estimation achieved using self-supervised completion with multi-view RGB stereo inputs. Requirements real-time fast inference speeds are met efficient, learned image feature projections. Furthermore, model trained large-scale, real-world dataset collected across variety diverse environments. We show how TerrainNet can also used costmap provide detailed framework integration into planning module. demonstrate through extensive comparison state-of-the-art baselines camera-only prediction. Finally, showcase effectiveness integrating within complete autonomous-driving stack by conducting vehicle test challenging scenario.