Reference-guided structure-aware deep sketch colorization for cartoons

• DOI: 10.1007/s41095-021-0228-6
• Publication Date: 2021-10-27T16:03:16Z
• Authors: Xueting Liu; Wenliang Wu; Chengze Li; Yifan Li; Huisi Wu
• Keywords: Electronic computers. Computer science; sketch colorization; 0202 electrical engineering, electronic engineering, information engineering; image style editing; reference-based image colorization; QA75.5-76.95; 02 engineering and technology; deep feature understanding

Abstract Digital cartoon production requires extensive manual labor to colorize sketches with visually pleasant color composition and color shading. During colorization, the artist usually takes an existing cartoon image as color guidance, particularly when colorizing related characters or an animation sequence. Reference-guided colorization is more intuitive than colorization with other hints, such as color points or scribbles, or text-based hints. Unfortunately, reference-guided colorization is challenging since the style of the colorized image should match the style of the reference image in terms of both global color composition and local color shading. In this paper, we propose a novel learning-based framework which colorizes a sketch based on a color style feature extracted from a reference color image. Our framework contains a color style extractor to extract the color feature from a color image, a colorization network to generate multi-scale output images by combining a sketch and a color feature, and a multi-scale discriminator to improve the reality of the output image. Extensive qualitative and quantitative evaluations show that our method outperforms existing methods, providing both superior visual quality and style reference consistency in the task of reference-based colorization.