SAPA: Similarity-Aware Point Affiliation for Feature Upsampling

• DOI: 10.48550/arxiv.2209.12866
• Publication Date: 2022-01-01
• Authors: Hao Lu; Wenze Liu; Zixuan Ye; Hongtao Fu; Yuliang Liu; Zhiguo Cao
• Keywords: Upsampling; Feature (linguistics); Similarity (geometry); Smoothness

We introduce point affiliation into feature upsampling, a notion that describes the of each upsampled to semantic cluster formed by local decoder points with similarity. By rethinking affiliation, we present generic formulation for generating upsampling kernels. The kernels encourage not only smoothness but also boundary sharpness in maps. Such properties are particularly useful some dense prediction tasks such as segmentation. key idea our is generate similarity-aware comparing similarity between encoder and spatially associated region features. In this way, can function cue inform points. To embody formulation, further instantiate lightweight operator, termed Similarity-Aware Point Affiliation (SAPA), investigate its variants. SAPA invites consistent performance improvements on number tasks, including segmentation, object detection, depth estimation, image matting. Code available at: https://github.com/poppinace/sapa