Self-Supervised Learning for Semi-Supervised Temporal Action Proposal

• DOI: 10.48550/arxiv.2104.03214
• Publication Date: 2021-01-01
• Authors: Xiang Wang; Shiwei Zhang; Zhiwu Qing; Yuanjie Shao; Changxin Gao; Nong Sang
• Keywords: Discriminative model; Supervised Learning; Feature (linguistics); Code (set theory); Pretext

Self-supervised learning presents a remarkable performance to utilize unlabeled data for various video tasks. In this paper, we focus on applying the power of self-supervised methods improve semi-supervised action proposal generation. Particularly, design an effective Semi-supervised Temporal Action Proposal (SSTAP) framework. The SSTAP contains two crucial branches, i.e., temporal-aware branch and relation-aware branch. improves model by introducing temporal perturbations, feature shift flip, in mean teacher defines pretext tasks, including masked reconstruction clip-order prediction, learn relation clues. By means, can better explore videos, discriminative abilities learned features. We extensively evaluate proposed THUMOS14 ActivityNet v1.3 datasets. experimental results demonstrate that significantly outperforms state-of-the-art even matches fully-supervised methods. Code is available at https://github.com/wangxiang1230/SSTAP.