Robust Training of Federated Models with Extremely Label Deficiency

• DOI: 10.48550/arxiv.2402.14430
• Publication Date: 2024-02-22
• Authors: Yonggang Zhang; Zhiqin Yang; Xinmei Tian; Nannan Wang; Tongliang Liu; Bo Han

Federated semi-supervised learning (FSSL) has emerged as a powerful paradigm for collaboratively training machine models using distributed data with label deficiency. Advanced FSSL methods predominantly focus on single model each client. However, this approach could lead to discrepancy between the objective functions of labeled and unlabeled data, resulting in gradient conflicts. To alleviate conflict, we propose novel twin-model paradigm, called Twin-sight, designed enhance mutual guidance by providing insights from different perspectives data. In particular, Twin-sight concurrently trains supervised function while an unsupervised function. synergy these two models, introduces neighbourhood-preserving constraint, which encourages preservation neighbourhood relationship among features extracted both models. Our comprehensive experiments four benchmark datasets provide substantial evidence that can significantly outperform state-of-the-art across various experimental settings, demonstrating efficacy proposed Twin-sight.