NeRFFaceLighting: Implicit and Disentangled Face Lighting Representation Leveraging Generative Prior in Neural Radiance Fields

• DOI: 10.1145/3597300
• Publication Date: 2023-05-16T12:05:45Z
• Authors: Kaiwen Jiang; Shu-Yu Chen; Hongbo Fu; Lin Gao
• Keywords: Representation; Generative model

3D-aware portrait lighting control is an emerging and promising domain, thanks to the recent advance of generative adversarial networks neural radiance fields. Existing solutions typically try decouple from geometry appearance for disentangled with explicit representation (e.g., Lambertian or Phong). However, they either are limited a constrained condition directional light) demand tricky-to-fetch dataset as supervision intrinsic compositions albedo). We propose NeRFFaceLighting explore implicit based on pretrained tri-plane address above limitations. approach this lighting-control problem by distilling shading original fused both (i.e., one tri-plane) their representations two tri-planes) conditional discriminator supervise effects. further carefully design regularization reduce ambiguity such decomposition enhance ability generalization unseen conditions. Moreover, our method can be extended enable real relighting. Through extensive quantitative qualitative evaluations, we demonstrate superior model compared alternative existing solutions.