Kernelized Heterogeneous Risk Minimization

• DOI: 10.48550/arxiv.2110.12425
• Publication Date: 2021-01-01
• Authors: Jiashuo Liu; Zheyuan Hu; Peng Cui; Bo Li; Zheyan Shen
• Keywords: Empirical risk minimization; Structural risk minimization; Minification; Kernel (algebra); Deep Neural Networks

The ability to generalize under distributional shifts is essential reliable machine learning, while models optimized with empirical risk minimization usually fail on non-$i.i.d$ testing data. Recently, invariant learning methods for out-of-distribution (OOD) generalization propose find causally relationships multi-environments. However, modern datasets are frequently multi-sourced without explicit source labels, rendering many inapplicable. In this paper, we Kernelized Heterogeneous Risk Minimization (KerHRM) algorithm, which achieves both the latent heterogeneity exploration and in kernel space, then gives feedback original neural network by appointing gradient direction. We theoretically justify our algorithm empirically validate effectiveness of extensive experiments.