DFMSD: Dual Feature Masking Stage-wise Knowledge Distillation for Object Detection

• DOI: 10.48550/arxiv.2407.13147
• Publication Date: 2024-07-18
• Authors: Zhourui Zhang; Jun Li; Zhijian Wu; Jifeng Shen; Jianhua Xu
• Keywords: Feature (linguistics); Backward masking

In recent years, current mainstream feature masking distillation methods mainly function by reconstructing selectively masked regions of a student network from the maps teacher network. these methods, attention mechanisms can help to identify spatially important and crucial object-aware channel clues, such that reconstructed features are encoded with sufficient discriminative representational power similar features. However, previous feature-masking address homogeneous knowledge without fully taking into account heterogeneous scenario. particular, huge discrepancy between frameworks within paradigm is detrimental masking, leading deteriorating this study, novel dual framework termed DFMSD proposed for object detection. More specifically, stage-wise adaptation learning module incorporated framework, thus model be progressively adapted models bridging gap networks. Furthermore, enhancement strategy combined adaptively strengthened improve reconstruction. addition, semantic alignment performed at each Feature Pyramid Network (FPN) layer networks generating consistent distributions. Our experiments detection task demonstrate promise our approach, suggesting outperforms both state-of-the-art methods.