Towards Unbiased Multi-label Zero-Shot Learning with Pyramid and Semantic Attention
FOS: Computer and information sciences
330
Multi-label Zero-shot learning
Computer Vision and Pattern Recognition (cs.CV)
Semantic Feature Space
Computer Science - Computer Vision and Pattern Recognition
Computational modeling
Image recognition
Pattern Recognition
Classification
Correlation
Semantics
004
03 medical and health sciences
0302 clinical medicine
Attention Mechanism
Task analysis
Feature extraction
Training
DOI:
10.48550/arxiv.2203.03483
Publication Date:
2023-01-01
AUTHORS (5)
ABSTRACT
Multi-label zero-shot learning extends conventional single-label zero-shot learning to a more realistic scenario that aims at recognizing multiple unseen labels of classes for each input sample. Existing works usually exploit attention mechanism to generate the correlation among different labels. However, most of them are usually biased on several major classes while neglect most of the minor classes with the same importance in input samples, and may thus result in overly diffused attention maps that cannot sufficiently cover minor classes. We argue that disregarding the connection between major and minor classes, i.e., correspond to the global and local information, respectively, is the cause of the problem. In this paper, we propose a novel framework of unbiased multi-label zero-shot learning, by considering various class-specific regions to calibrate the training process of the classifier. Specifically, Pyramid Feature Attention (PFA) is proposed to build the correlation between global and local information of samples to balance the presence of each class. Meanwhile, for the generated semantic representations of input samples, we propose Semantic Attention (SA) to strengthen the element-wise correlation among these vectors, which can encourage the coordinated representation of them. Extensive experiments on the large-scale multi-label zero-shot benchmarks NUS-WIDE and Open-Image demonstrate that the proposed method surpasses other representative methods by significant margins.
SUPPLEMENTAL MATERIAL
Coming soon ....
REFERENCES ()
CITATIONS ()
EXTERNAL LINKS
PlumX Metrics
RECOMMENDATIONS
FAIR ASSESSMENT
Coming soon ....
JUPYTER LAB
Coming soon ....