We present a descriptor, called fully convolutional self-similarity (FCSS), for dense semantic correspondence. To robustly match points among different instances within the same object class, we formulate FCSS using local (LSS) network. In contrast to existing CNN-based descriptors, is inherently insensitive intra-class appearance variations because of its LSS-based structure, while maintaining precise localization ability deep neural networks. The sampling patterns structure and measure are...

We propose a novel cost aggregation network, called Cost Aggregation Transformers (CATs), to find dense correspondences between semantically similar images with additional challenges posed by large intra-class appearance and geometric variations. is highly important process in matching tasks, which the accuracy depends on quality of its output. Compared hand-crafted or CNN-based methods addressing aggregation, that either lacks robustness severe deformations inherit limitation CNNs fail...

We present semantic attribute matching networks (SAM-Net) for jointly establishing correspondences and transferring attributes across semantically similar images, which intelligently weaves the advantages of two tasks while overcoming their limitations. SAM-Net accomplishes this through an iterative process reliable by reducing discrepancy between images synthesizing transferred using learned correspondences. To learn weak supervisions in form image pairs, we a loss based on similarity...

We present a novel framework for contrastive learning of pixel-level representation using only unlabeled video. Without the need ground-truth annotation, our method is capable collecting well-defined positive correspondences by measuring their confidences and negative ones appropriately adjusting hardness during training. This allows us to suppress adverse impact ambiguous matches prevent trivial solution from being yielded too hard or easy samples. To accomplish this, we incorporate three...

Convolutional neural networks (CNNs) based approaches for semantic alignment and object landmark detection have improved their performance significantly. Current efforts the two tasks focus on addressing lack of massive training data through weakly- or unsupervised learning frameworks. In this paper, we present a joint approach obtaining dense correspondences discovering landmarks from semantically similar images. Based key insight that can mutually provide supervisions to each other, our...

We present a descriptor, called fully convolutional self-similarity (FCSS), for dense semantic correspondence. To robustly match points among different instances within the same object class, we formulate FCSS using local (LSS) network. In contrast to existing CNN-based descriptors, is inherently insensitive intra-class appearance variations because of its LSS-based structure, while maintaining precise localization ability deep neural networks. The sampling patterns structure and measure are...

We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes every segments. Our networks accomplish this through multi-task fusion approach based on two types of attention modules to explore semantic correlations between the videos. The proposed consist feature embedding inference stochastically leverage inferred representations. Additionally, we design new center loss function learns representations by enforcing...

This paper presents a novel deep architecture for weakly-supervised temporal action localization that not only generates segment-level responses but also propagates to the neighborhood in form of graph Laplacian regularization. Specifically, our approach consists two sub-modules; class activation module estimate score map over time through classifiers, and regularization refine estimated by solving quadratic programming problem with predicted semantic affinities. Since these modules are...

Recent vision-based reinforcement learning (RL) methods have found extracting high-level features from raw pixels with self-supervised to be effective in policies. However, these focus on global representations of images, and disregard local spatial structures present the consecutively stacked frames. In this paper, we propose a novel approach, termed Paired Similarity Representation Learning (PSRL) for effectively encoding an unsupervised manner. Given input frames, latent volumes are first...

Existing building recognition methods, exemplified by BRAILS, utilize supervised learning to extract information from satellite and street-view images for classification segmentation. However, each task module requires human-annotated data, hindering the scalability robustness regional variations annotation imbalances. In response, we propose a new zero-shot workflow attribute extraction that utilizes large-scale vision language models mitigate reliance on external annotations. The proposed...

Existing pipelines of semantic correspondence commonly include extracting high-level features for the invariance against intra-class variations and background clutters. This architecture, however, inevitably results in a low-resolution matching field that additionally requires an ad-hoc interpolation process as post-processing converting it into high-resolution one, certainly limiting overall performance results. To overcome this, inspired by recent success implicit neural representation, we...

While recognizing human actions and surrounding scenes addresses different aspects of video understanding, they have strong correlations that can be used to complement the singular information each other. In this paper, we propose an approach for joint action scene recognition is formulated in end-to-end learning framework based on temporal attention techniques fusion them. By applying modules generic feature network, features are extracted efficiently, then composed a single vector through...

We present semantic attribute matching networks (SAM-Net) for jointly establishing correspondences and transferring attributes across semantically similar images, which intelligently weaves the advantages of two tasks while overcoming their limitations. SAM-Net accomplishes this through an iterative process reliable by reducing discrepancy between images synthesizing transferred using learned correspondences. To learn weak supervisions in form image pairs, we a loss based on similarity...

This paper presents a deep architecture, called pyramidal semantic correspondence networks (PSCNet), that estimates locally-varying affine transformation fields across semantically similar images. To deal with large appearance and shape variations commonly exist among different instances within the same object category, we leverage model where are progressively estimated in coarse-to-fine manner so smoothness constraint is naturally imposed. Different from previous methods which directly...

This paper presents a deep architecture for dense semantic correspondence, called pyramidal affine regression networks (PARN), that estimates locally-varying transformation fields across images. To deal with intra-class appearance and shape variations commonly exist among different instances within the same object category, we leverage model where are progressively estimated in coarse-to-fine manner so smoothness constraint is naturally imposed networks. PARN residual transformations at each...

Existing building recognition methods, exemplified by BRAILS, utilize supervised learning to extract information from satellite and street-view images for classification segmentation. However, each task module requires human-annotated data, hindering the scalability robustness regional variations annotation imbalances. In response, we propose a new zero-shot workflow attribute extraction that utilizes large-scale vision language models mitigate reliance on external annotations. The proposed...

We present a novel fusion scheme between multiple intermediate convolutional features within neurual network (CNN) for dense correspondence estimation. In contrast to existing CNN-based descriptors that utilize single activation, our approach jointly uses of CNN through the attention weight balances contribution each features. formulate overall as two sub-networks, and network. The is designed provide matching costs while learn optimal them. These networks are learned in joint manner boost...

Convolutional neural networks (CNNs) based approaches for semantic alignment and object landmark detection have improved their performance significantly. Current efforts the two tasks focus on addressing lack of massive training data through weakly- or unsupervised learning frameworks. In this paper, we present a joint approach obtaining dense correspondences discovering landmarks from semantically similar images. Based key insight that can mutually provide supervisions to each other, our...