Towards Mitigating Architecture Overfitting in Dataset Distillation

• DOI: 10.48550/arxiv.2309.04195
• Publication Date: 2023-01-01
• Authors: Xuyang Zhong; Chen Liu
• Keywords: Overfitting; Generality; Adaptability; Training set

Dataset distillation methods have demonstrated remarkable performance for neural networks trained with very limited training data. However, a significant challenge arises in the form of architecture overfitting: distilled data synthesized by specific network (i.e., network) generates poor when other architectures test networks). This paper addresses this issue and proposes series approaches both designs schemes which can be adopted together to boost generalization across different on We conduct extensive experiments demonstrate effectiveness generality our methods. Particularly, various scenarios involving sizes data, achieve comparable or superior existing using larger capacities.