Multiface: A Dataset for Neural Face Rendering

• DOI: 10.48550/arxiv.2207.11243
• Publication Date: 2022-01-01
• Authors: Cheng-hsin Wuu; Ningyuan Zheng; Scott Ardisson; Rohan Bali; Danielle Belko; Eric Brockmeyer; Lucas Evans; Timothy Godisart; Hyowon Ha; Alexander Hypes; Taylor Koska; Steven Krenn; Stephen Lombardi; Xiaomin Luo; Kevyn McPhail; Laura Millerschoen; Michal Perďoch; Mark Pitts; Alexander Richard; Jason Saragih; Junko Saragih; Takaaki Shiratori; Tomas Simon; Matt Stewart; Autumn Trimble; Xinshuo Weng; David Whitewolf; Chenglei Wu; Shoou-I Yu; Yaser Sheikh
• Keywords: FOS: Computer and information sciences; Computer Science - Graphics; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Graphics (cs.GR)

Photorealistic avatars of human faces have come a long way in recent years, yet research along this area is limited by lack publicly available, high-quality datasets covering both, dense multi-view camera captures, and rich facial expressions the captured subjects. In work, we present Multiface, new multi-view, high-resolution face dataset collected from 13 identities at Reality Labs Research for neural rendering. We introduce Mugsy, large scale multi-camera apparatus to capture synchronized videos performance. The goal Multiface close gap accessibility high quality data academic community enable VR telepresence. Along with release dataset, conduct ablation studies on influence different model architectures toward model's interpolation capacity novel viewpoint expressions. With conditional VAE serving as our baseline, found that adding spatial bias, texture warp field, residual connections improves performance view synthesis. Our code available at: https://github.com/facebookresearch/multiface