Multivariate-Information Adversarial Ensemble for Scalable Joint Distribution Matching

• DOI: 10.48550/arxiv.1907.03426
• Publication Date: 2019-01-01
• Authors: Chen, Ziliang; Yang, Zhanfu; Wang, Xiaoxi; Liang, Xiaodan; Yan, Xiaopeng; Li, Guanbin; Lin, Liang
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Statistics - Machine Learning; 0202 electrical engineering, electronic engineering, information engineering; Machine Learning (stat.ML); 02 engineering and technology; Machine Learning (cs.LG)

A broad range of cross-$m$-domain generation researches boil down to matching a joint distribution by deep generative models (DGMs). Hitherto algorithms excel in pairwise domains while as $m$ increases, remain struggling to scale themselves to fit a joint distribution. In this paper, we propose a domain-scalable DGM, i.e., MMI-ALI for $m$-domain joint distribution matching. As an $m$-domain ensemble model of ALIs \cite{dumoulin2016adversarially}, MMI-ALI is adversarially trained with maximizing Multivariate Mutual Information (MMI) w.r.t. joint variables of each pair of domains and their shared feature. The negative MMIs are upper bounded by a series of feasible losses that provably lead to matching $m$-domain joint distributions. MMI-ALI linearly scales as $m$ increases and thus, strikes a right balance between efficacy and scalability. We evaluate MMI-ALI in diverse challenging $m$-domain scenarios and verify its superiority.<br/>ICML-19<br/>