MM-Gaussian: 3D Gaussian-based Multi-modal Fusion for Localization and Reconstruction in Unbounded Scenes

• DOI: 10.48550/arxiv.2404.04026
• Publication Date: 2024-04-05
• Authors: Chenyang Wu; Yifan Duan; Xinran Zhang; Sheng Yu; Jianmin Ji; Yanyong Zhang
• Keywords: FOS: Computer and information sciences; Computer Science - Robotics; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; Robotics (cs.RO)

Localization and mapping are critical tasks for various applications such as autonomous vehicles robotics. The challenges posed by outdoor environments present particular complexities due to their unbounded characteristics. In this work, we MM-Gaussian, a LiDAR-camera multi-modal fusion system localization in scenes. Our approach is inspired the recently developed 3D Gaussians, which demonstrate remarkable capabilities achieving high rendering quality fast speed. Specifically, our fully utilizes geometric structure information provided solid-state LiDAR address problem of inaccurate depth encountered when relying solely on visual solutions unbounded, scenarios. Additionally, utilize Gaussian point clouds, with assistance pixel-level gradient descent, exploit color photos, thereby realistic effects. To further bolster robustness system, designed relocalization module, assists returning correct trajectory event failure. Experiments conducted multiple scenarios effectiveness method.