We present a novel approach for image completion that results in images are both locally and globally consistent. With fully-convolutional neural network, we can complete of arbitrary resolutions by filling-in missing regions any shape. To train this network to be consistent, use global local context discriminators trained distinguish real from completed ones. The discriminator looks at the entire assess if it is coherent as whole, while only small area centered region ensure consistency...

Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal the large number potential combination stochastic sampling training set an aggressive mining strategy biased towards patches that are...

We present a novel technique to automatically colorize grayscale images that combines both global priors and local image features. Based on Convolutional Neural Networks, our deep network features fusion layer allows us elegantly merge information dependent small patches with computed using the entire image. The framework, including as well colorization model, is trained in an end-to-end fashion. Furthermore, architecture can process of any resolution, unlike most existing approaches based...

In this paper, we analyze the fashion of clothing a large social website. Our goal is to learn and predict how fashionable person looks on photograph suggest subtle improvements user could make improve her/his appeal. We propose Conditional Random Field model that jointly reasons about several fashionability factors such as type outfit garments wearing, user, photograph's setting (e.g., scenery behind user), score. Importantly, our able give rich feedback back conveying which or even she/he...

In this paper, we present a novel technique to simplify sketch drawings based on learning series of convolution operators. contrast existing approaches that require vector images as input, allow the more general and challenging input rough raster sketches such those obtained from scanning pencil sketches. We convert into simplified version which is then amendable for vectorization. This all done in fully automatic way without user intervention. Our model consists convolutional neural network...

We propose a novel approach for learning features from weakly-supervised data by joint ranking and classification. In order to exploit with weak labels, we jointly train feature extraction network loss classification cross-entropy loss. obtain high-quality compact discriminative few parameters, learned on relatively small datasets without additional annotations. This enables us tackle tasks specialized images not very similar the more generic ones in existing fully-supervised datasets. show...

We introduce a novel approach to automatically recover 3D human pose from single image. Most previous work follows pipelined approach: initially, set of 2D features such as edges, joints or silhouettes are detected in the image, and then these observations used infer pose. Solving two problems separately may lead erroneous poses when feature detector has performed poorly. In this paper, we address issue by jointly solving both detection inference problems. For purpose, propose Bayesian...

Markerless 3D human pose detection from a single image is severely underconstrained problem because different poses can have similar projections. In order to handle this ambiguity, current approaches rely on prior shape models that only be correctly adjusted if 2D features are accurately detected. Unfortunately, although part detector algorithms shown promising results, they not yet accurate enough guarantee complete disambiguation of the inferred shape. paper, we introduce novel approach...

We present an integral framework for training sketch simplification networks that convert challenging rough sketches into clean line drawings. Our approach augments a network with discriminator network, both jointly so the discerns whether drawing is real data or output of which, in turn, tries to fool it. This has two major advantages: first, because learns structure drawings, it encourages be more similar appearance sketches. Second, we can also train additional unsupervised data: by...

It is common in graphic design humans visually arrange various elements according to their intent and semantics. For example, a title text almost always appears on top of other document. In this work, we generate layouts that can flexibly incorporate such semantics, either specified implicitly or explicitly by user. We optimize using the latent space an off-the-shelf layout generation model, allowing our approach be complementary used with existing models. Our builds generative model based...

Controllable layout generation aims at synthesizing plausible arrangement of element bounding boxes with optional constraints, such as type or position a specific element. In this work, we try to solve broad range tasks in single model that is based on discrete state-space diffusion models. Our model, named Lay-outDM, naturally handles the structured data representation and learns progressively infer noiseless from initial input, where corruption process by modality-wise diffusion. For...

In this paper we propose a novel framework for learning local image descriptors in discriminative manner. For purpose explore siamese architecture of Deep Convolutional Neural Networks (CNN), with Hinge embedding loss on the L2 distance between descriptors. Since uses pairs rather than single patches to train, there exist large number positive samples and an exponential negative samples. We space stochastic sampling training set, combination aggressive mining strategy over both which denote...

The remastering of vintage film comprises a diversity sub-tasks including super-resolution, noise removal, and contrast enhancement which aim to restore the deteriorated medium its original state. Additionally, due technical limitations time, most is either recorded in black white, or has low quality colors, for colorization becomes necessary. In this work, we propose single framework tackle entire task semi-interactively. Our work based on temporal convolutional neural networks with...

Flat filling is a critical step in digital artistic content creation with the objective of line arts flat colors. We present deep learning framework for user-guided art that can compute "influence areas" user color scribbles, i.e., areas where scribbles should propagate and influence. This explicitly controls such scribble influence artists to manipulate colors image details avoid leakage/contamination between simultaneously, leverages data-driven generation facilitate creation. based on...

We tackle the problem of multi-label classification fashion images, learning from noisy data with minimal human supervision. present a new dataset full body poses, each set 66 binary labels corresponding to information about garments worn in image obtained an automatic manner. As automatically-collected contain significant noise, we manually correct for small subset data, and use these further training evaluation. build upon recent approach that both cleans learns classify, introduce simple...

In this work, we perform an experimental analysis of the differences both how humans and machines see distinguish fashion styles. For purpose, propose expert-curated new dataset for style prediction, which consists 14 different styles each with roughly 1,000 images worn outfits. The dataset, a total 13,126 images, captures diversity complexity modern We extensive by benchmarking wide variety classification networks, also in-depth user study fashion-savvy fashion-naïve users. Our results...

Vector line art plays an important role in graphic design, however, it is tedious to manually create. We introduce a general framework produce drawings from wide variety of images, by learning mapping raster image space vector space. Our approach based on recurrent neural network that draws the lines one one. A differentiable rasterization module allows for training with only supervised data. use dynamic window around virtual pen while drawing lines, implemented proposed aligned cropping and...

We present an interactive approach for inking , which is the process of turning a pencil rough sketch into clean line drawing. The approach, we call Smart Inker consists several "smart" tools that intuitively react to user input, while guided by input sketch, efficiently and naturally connect lines, erase shading, fine-tune drawing output. Our data-driven: are based on fully convolutional networks, train exploit both edits inaccurate produce accurate drawings, allowing high-performance...

We propose a novel data-driven approach for automatically detecting and completing gaps in line drawings with Convolutional Neural Network. In the case of existing inpainting approaches natural images, masks indicating missing regions are generally required as input. Here, we show that have enough structures can be learned by CNN to allow automatic detection completion without any such Thus, our method find complete them user interaction. Furthermore, realistically conserves thickness...

We present an approach for the detection of buildings in multispectral satellite images. Unlike 3-channel RGB images, imagery contains additional channels corresponding to different wavelengths. Approaches that do not use all are unable fully exploit these images optimal performance. Furthermore, care must be taken due large bias classes, e.g., most Earth is covered water and thus it will dominant Our consists training a Convolutional Neural Network (CNN) from scratch classify image patches...

Line art plays a fundamental role in illustration and design, allows for iteratively polishing designs. However, as they lack color, can have issues conveying final In this work, we propose an interactive colorization approach based on conditional generative adversarial network that takes both the line color hints inputs to produce high-quality colorized image. Our is U-net architecture with multi-discriminator framework. We Concatenation Spatial Attention module able generate more...