Sketch-to-Art: Synthesizing Stylized Art Images From Sketches

• DOI: 10.48550/arxiv.2002.12888
• Publication Date: 2020-01-01
• Authors: Liu, Bingchen; Song, Kunpeng; Elgammal, Ahmed
• Keywords: FOS: Computer and information sciences; Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Computer Science - Computer Vision and Pattern Recognition; 0202 electrical engineering, electronic engineering, information engineering; FOS: Electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Electrical Engineering and Systems Science - Image and Video Processing

We propose a new approach for synthesizing fully detailed art-stylized images from sketches. Given a sketch, with no semantic tagging, and a reference image of a specific style, the model can synthesize meaningful details with colors and textures. The model consists of three modules designed explicitly for better artistic style capturing and generation. Based on a GAN framework, a dual-masked mechanism is introduced to enforce the content constraints (from the sketch), and a feature-map transformation technique is developed to strengthen the style consistency (to the reference image). Finally, an inverse procedure of instance-normalization is proposed to disentangle the style and content information, therefore yields better synthesis performance. Experiments demonstrate a significant qualitative and quantitative boost over baselines based on previous state-of-the-art techniques, adopted for the proposed process.<br/>24 pages<br/>