Video Generation from Single Semantic Label Map

• DOI: 10.48550/arxiv.1903.04480
• Publication Date: 2019-01-01
• Authors: Junting Pan; Chengyu Wang; Xu Jia; Jing Shao; Lu Sheng; Junjie Yan; Xiaogang Wang
• Keywords: Optical Flow; Sequence (biology)

This paper proposes the novel task of video generation conditioned on a SINGLE semantic label map, which provides good balance between flexibility and quality in process. Different from typical end-to-end approaches, model both scene content dynamics single step, we propose to decompose this difficult into two sub-problems. As current image methods do better than terms detail, synthesize high by only generating first frame. Then animate based its meaning obtain temporally coherent video, giving us excellent results overall. We employ cVAE for predicting optical flow as beneficial intermediate step generate sequence initial A map is integrated prediction module achieve major improvements image-to-video Extensive experiments Cityscapes dataset show that our method outperforms all competing methods.