HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion

• DOI: 10.1145/3592415
• Publication Date: 2023-07-26T15:47:45Z
• Authors: Mustafa Işık; Martin Rünz; Markos Georgopoulos; Taras Khakhulin; Jonathan Starck; Lourdes Agapito; Matthias Nießner
• Keywords: High fidelity

Representing human performance at high-fidelity is an essential building block in diverse applications, such as film production, computer games or videoconferencing. To close the gap to production-level quality, we introduce HumanRF, a 4D dynamic neural scene representation that captures full-body appearance motion from multi-view video input, and enables playback novel, unseen viewpoints. Our novel acts encoding fine details high compression rates by factorizing space-time into temporal matrix-vector decomposition. This allows us obtain temporally coherent reconstructions of actors for long sequences, while representing high-resolution even context challenging motion. While most research focuses on synthesizing resolutions 4MP lower, address challenge operating 12MP. this end, ActorsHQ, dataset provides 12MP footage 160 cameras 16 sequences with high-fidelity, per-frame mesh reconstructions. We demonstrate challenges emerge using data show our newly introduced HumanRF effectively leverages data, making significant step towards quality view synthesis.