VR content creation and exploration with deep learning: A survey

• DOI: 10.1007/s41095-020-0162-z
• Publication Date: 2020-03-23T09:03:35Z
• Authors: Miao Wang; Xu-Quan Lyu; Yi-Jun Li; Fang-Lue Zhang
• Keywords: Electronic computers. Computer science; 0202 electrical engineering, electronic engineering, information engineering; virtual reality; deep learning; QA75.5-76.95; 02 engineering and technology; 360° image and video virtual content; neural networks

Abstract Virtual reality (VR) offers an artificial, computer generated simulation of a real life environment. It originated in the 1960s and has evolved to provide increasing immersion, interactivity, imagination, and intelligence. Because deep learning systems are able to represent and compose information at various levels in a deep hierarchical fashion, they can build very powerful models which leverage large quantities of visual media data. Intelligence of VR methods and applications has been significantly boosted by the recent developments in deep learning techniques. VR content creation and exploration relates to image and video analysis, synthesis and editing, so deep learning methods such as fully convolutional networks and general adversarial networks are widely employed, designed specifically to handle panoramic images and video and virtual 3D scenes. This article surveys recent research that uses such deep learning methods for VR content creation and exploration. It considers the problems involved, and discusses possible future directions in this active and emerging research area.