DepthSplat: Connecting Gaussian Splatting and Depth

• DOI: 10.48550/arxiv.2410.13862
• Publication Date: 2024-10-17
• Authors: Haofei Xu; Songyou Peng; Fangjinhua Wang; Hermann Blum; Dániel Baráth; Andreas Geiger; Marc Pollefeys
• Keywords: FOS: Computer and information sciences; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition

Gaussian splatting and single/multi-view depth estimation are typically studied in isolation. In this paper, we present DepthSplat to connect study their interactions. More specifically, first contribute a robust multi-view model by leveraging pre-trained monocular features, leading high-quality feed-forward 3D reconstructions. We also show that can serve as an unsupervised pre-training objective for learning powerful models from large-scale unlabelled datasets. validate the synergy between through extensive ablation cross-task transfer experiments. Our achieves state-of-the-art performance on ScanNet, RealEstate10K DL3DV datasets terms of both novel view synthesis, demonstrating mutual benefits connecting tasks. code, models, video results available at https://haofeixu.github.io/depthsplat/.