Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?

• DOI: 10.48550/arxiv.2203.09081
• Publication Date: 2022-01-01
• Authors: Yibo Yang; Shixiang Chen; Xiangtai Li; Liang Xie; Zhouchen Lin; Dacheng Tao
• Keywords: Margin classifier; Simplex; Linear classifier

Modern deep neural networks for classification usually jointly learn a backbone representation and linear classifier to output the logit of each class. A recent study has shown phenomenon called collapse that within-class means features vectors converge vertices simplex equiangular tight frame (ETF) at terminal phase training on balanced dataset. Since ETF geometric structure maximally separates pair-wise angles all classes in classifier, it is natural raise question, why do we spend an effort when know its optimal structure? In this paper, potential learning network with randomly initialized as fixed during training. Our analytical work based layer-peeled model indicates feature naturally leads state even dataset imbalanced among classes. We further show case cross entropy (CE) loss not necessary can be replaced by simple squared shares same global optimality but enjoys better convergence property. experimental results our method able bring significant improvements faster multiple datasets.