Sketch-based image retrieval (SBIR) is a challenging task due to the ambiguity inherent in sketches when compared with photos. In this paper, we propose novel convolutional neural network based on Siamese for SBIR. The main idea pull output feature vectors closer input sketch-image pairs that are labeled as similar, and push them away if irrelevant. This achieved by jointly tuning two networks which linked one loss function. Experimental results Flickr15K demonstrate proposed method offers...

We propose a perceptual grouping framework that organizes image edges into meaningful structures and demonstrate its usefulness on various computer vision tasks. Our grouper formulates edge as graph partition problem, where learning to rank method is developed encode probabilities of candidate pairs. In particular, RankSVM employed for the first time combine multiple Gestalt principles cue grouping. Afterwards, an based object proposal measure introduced yields proposals comparable...

This paper studies the problem of zero-short sketch-based image retrieval (ZS-SBIR), however with two significant differentiators to prior art (i) we tackle all variants (inter-category, intracategory, and cross datasets) ZS-SBIR just one network ("everything"), (ii) would really like understand how this sketch-photo matching operates ("explainable"). Our key innovation lies realization that such a cross-modal could be reduced comparisons groups local patches - akin seasoned "bag-of-words"...

We provide a new fully automatic framework to analyze facial action units, the fundamental building blocks of expression enumerated in Paul Ekman's coding system (FACS). The units examined here include upper muscle movements such as inner eyebrow raise, eye widening, and so forth, which combine form expressions. Although prior methods have obtained high recognition rates for recognizing these either use manually preprocessed image sequences or require human specification features; thus, they...

In this paper, for the first time, we investigate problem of generating 3D shapes from professional 2D sketches via deep learning. We target done by artists, as these are likely to contain more details than ones produced novices, and thus reconstruction such poses a higher demand on level detail in reconstructed models. This is importantly different previous work, where training testing was conducted either synthetic or novices. Novices often depict that physically unrealistic, while models...

Despite the rapid advancements in text-to-image (T2I) synthesis, enabling precise visual control remains a significant challenge. Existing works attempted to incorporate multi-facet controls (text and sketch), aiming enhance creative over generated images. However, our pilot study reveals that expressive power of humans far surpasses capabilities current methods. Users desire more versatile approach can accommodate their diverse intents, ranging from controlling individual subjects...

We investigate the problem of stroke-level sketch segmentation, which is to train machines assign strokes with semantic part labels given a input sketch. Solving segmentation opens door for fine-grained interpretation, can benefit many novel sketch-based applications, including recognition and image retrieval. In this paper, we treat as seqence-to-sequence generation problem, reccurent nueral networks (RNN)-based model SketchSegNet presented translate sequence into thier labels. addition,...

In this paper we study, for the first time, problem of fine-grained sketch-based 3D shape retrieval. We advocate use sketches as a input modality to retrieve shapes at instance-level - e.g., given sketch chair, set out specific chair from gallery all chairs. Fine-grained retrieval (FG-SBSR) has not been possible till now due lack datasets that exhibit one-to-one sketch-3D correspondences. The key contribution is two new datasets, consisting total 4,680 pairings object categories. Even with...

We investigate the problem of stroke-level sketch segmentation, which is to automatically assign strokes a given with semantic labels. Solving segmentation opens door for fine-grained interpretation, can benefit many novel sketch-based applications, including recognition and image retrieval. In this paper, we propose an approach multi-class by considering it as sequence-to-sequence generation problem. Specifically, end-to-end learned network <italic...

Sketch is used for rendering the visual world since prehistoric times, and has become ubiquitous nowadays with increasing availability of touchscreens on portable devices. However, how to automatically map images sketches, a problem that profound implications applications such as sketch-based image retrieval, still remains open. In this paper, we propose novel method draws sketch from single natural image. extraction posed within an unified contour grouping framework, where perceptual first...

The key challenge in designing a sketch representation lies with handling the abstract and iconic nature of sketches. Existing work predominantly utilizes either, (i) pixelative format that treats sketches as natural images employing off-the-shelf CNN-based networks, or (ii) an elaborately designed vector leverages structural information drawing orders using sequential RNN-based methods. While lacks intuitive exploitation cues, are absent most cases limiting their practical usage. Hence,...

Polymer materials with multiple stimuli-responsive properties have demonstrated many potential and practical applications. By covalently introducing spiropyran (SP1) spirothiopyran (STP) into the polyurethane backbone, photochromic, mechanochromic, thermally discolored polymer been prepared. In this work, we report for first time that white light (violet, blue, green light) above a certain intensity can activate STP to color. Based on discovery, SP1 exhibit reversible three-color changes...

Humans tend to understand image scene by recognizing visual elements, then conjecturing and inferring based on them, hence are able search relevant images. In this paper, we concern about the problem of complex retrieval reasoning dense captions, which is similar way human perception for searching Specifically, transform into a captioning graph matching issue using structured language descriptions retrieval. Experimental results novel proposed large-scale content-based dataset demonstrate...

User intention understanding from text is an important task in NLP. In this paper, we study the problem of phone-changing prediction. And propose a novel feature extraction method, which selects most representative feature, to represent user's scratch. Then adopt supervised learning approach, that train SVM classifier, for addition, dataset scratches and their corresponding labels are collected real network environment. The experimental results validate effectiveness our proposed approach.

Sketches are distinctly different to photos. They highly abstract and exhibit a severe lack of visual cues. Prior works have therefore explored additional traits unique sketches help recognition such as stroke ordering. In this paper, we pioneer in studying the role structure sketches, for task sketch recognition. particular, propose novel graph representation specifically designed which follows inherent hierarchical relationship (segment-stroke-sketch") sketching elements. By conforming...

Traffic sign recognition (TSR) is an indispensable component for vision-based system of self-driving car. Promising results have been achieved which especially benefit from the rapid development deep neural networks recently. However, there are few works focusing on algorithms' performances towards different complex conditions, such as weather and viewpoint variations. In this paper, we propose a new real-world TSR dataset, dataset with several fine-grained conditions fine labeled involving...

Image-to-sketch translation is to learn the mapping between an image and a corresponding human drawn sketch. Machine can be trained mimic drawing process using training set of aligned image-sketch pairs. However, collect such paired data quite expensive or even unavailable for many cases since sketches exhibit various level abstractness preferences. Hence we present approach learning image-to-sketch network via unpaired examples. A network, which translate representation in latent space...