Debiased Self-Training for Semi-Supervised Learning

• DOI: 10.48550/arxiv.2202.07136
• Publication Date: 2022-01-01
• Authors: Baixu Chen; Junguang Jiang; Ximei Wang; Pengfei Wan; Jianmin Wang; Mingsheng Long
• Keywords: Benchmark (surveying); Labeled data; Popularity; Training set; Supervised Learning

Deep neural networks achieve remarkable performances on a wide range of tasks with the aid large-scale labeled datasets. Yet these datasets are time-consuming and labor-exhaustive to obtain realistic tasks. To mitigate requirement for data, self-training is widely used in semi-supervised learning by iteratively assigning pseudo labels unlabeled samples. Despite its popularity, well-believed be unreliable often leads training instability. Our experimental studies further reveal that bias arises from both problem itself inappropriate potentially incorrect labels, which accumulates error iterative process. reduce above bias, we propose Debiased Self-Training (DST). First, generation utilization decoupled two parameter-independent classifier heads avoid direct accumulation. Second, estimate worst case where labeling function accurate samples, yet makes as many mistakes possible We then adversarially optimize representations improve quality avoiding case. Extensive experiments justify DST achieves an average improvement 6.3% against state-of-the-art methods standard benchmark 18.9%$ FixMatch 13 diverse Furthermore, can seamlessly adapted other help stabilize their balance performance across classes cases scratch finetuning pre-trained models.