VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data

• DOI: 10.48550/arxiv.2006.11941
• Publication Date: 2020-01-01
• Authors: Ma, Chao; Tschiatschek, Seb...; Hernandez-Lobato,...; Turner, Richard; Zhang, Cheng
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; 102019 Machine Learning; Statistics - Machine Learning; 102001 Artificial intelligence; 102001 Artificial Intelligence; Machine Learning (stat.ML); 102019 Machine learning; 01 natural sciences; 0105 earth and related environmental sciences; Machine Learning (cs.LG)

Deep generative models often perform poorly in real-world applications due to the heterogeneity of natural data sets. Heterogeneity arises from data containing different types of features (categorical, ordinal, continuous, etc.) and features of the same type having different marginal distributions. We propose an extension of variational autoencoders (VAEs) called VAEM to handle such heterogeneous data. VAEM is a deep generative model that is trained in a two stage manner such that the first stage provides a more uniform representation of the data to the second stage, thereby sidestepping the problems caused by heterogeneous data. We provide extensions of VAEM to handle partially observed data, and demonstrate its performance in data generation, missing data prediction and sequential feature selection tasks. Our results show that VAEM broadens the range of real-world applications where deep generative models can be successfully deployed.