gsanet semantic segmentation with global and selective attention

• DOI: 10.48550/arxiv.2003.00830
• Publication Date: 2020-10-01
• Authors: Jungwon Lee; Qingfeng Liu; Dongwoon Bai; Mostafa El-Khamy
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Computer Science - Computer Vision and Pattern Recognition; Machine Learning (stat.ML); Electrical Engineering and Systems Science - Image and Video Processing; Machine Learning (cs.LG); 03 medical and health sciences; 0302 clinical medicine; Statistics - Machine Learning; FOS: Electrical engineering, electronic engineering, information engineering

This paper proposes a novel deep learning architecture for semantic segmentation. The proposed Global and Selective Attention Network (GSANet) features Atrous Spatial Pyramid Pooling (ASPP) with a novel sparsemax global attention and a novel selective attention that deploys a condensation and diffusion mechanism to aggregate the multi-scale contextual information from the extracted deep features. A selective attention decoder is also proposed to process the GSA-ASPP outputs for optimizing the softmax volume. We are the first to benchmark the performance of semantic segmentation networks with the low-complexity feature extraction network (FXN) MobileNetEdge, that is optimized for low latency on edge devices. We show that GSANet can result in more accurate segmentation with MobileNetEdge, as well as with strong FXNs, such as Xception. GSANet improves the state-of-art semantic segmentation accuracy on both the ADE20k and the Cityscapes datasets.