Gaussian Splatting Decoder for 3D-aware Generative Adversarial Networks

• DOI: 10.48550/arxiv.2404.10625
• Publication Date: 2024-04-16
• Authors: Florian Barthel; Arian Beckmann; Wieland Morgenstern; Anna Hilsmann; Peter Eisert
• Keywords: Generative adversarial network

NeRF-based 3D-aware Generative Adversarial Networks (GANs) like EG3D or GIRAFFE have shown very high rendering quality under large representational variety. However, with Neural Radiance Fields poses challenges for 3D applications: First, the significant computational demands of NeRF preclude its use on low-power devices, such as mobiles and VR/AR headsets. Second, implicit representations based neural networks are difficult to incorporate into explicit scenes, VR environments video games. Gaussian Splatting (3DGS) overcomes these limitations by providing an representation that can be rendered efficiently at frame rates. In this work, we present a novel approach combines GANs flexibility advantages 3DGS. By training decoder maps attributes, integrate diversity ecosystem first time. Additionally, our allows resolution GAN inversion real-time editing scenes.