Discovering Hidden Factors of Variation in Deep Networks

• DOI: 10.48550/arxiv.1412.6583
• Publication Date: 2014-01-01
• Authors: Brian Cheung; Jesse A. Livezey; Arjun K. Bansal; Bruno A. Olshausen
• Keywords: MNIST database; Variation (astronomy); Handwriting; Regularization

Deep learning has enjoyed a great deal of success because its ability to learn useful features for tasks such as classification. But there been less exploration in the factors variation apart from classification signal. By augmenting autoencoders with simple regularization terms during training, we demonstrate that standard deep architectures can discover and explicitly represent beyond those relevant categorization. We introduce cross-covariance penalty (XCov) method disentangle like handwriting style digits subject identity faces. this on MNIST handwritten digit database, Toronto Faces Database (TFD) Multi-PIE dataset by generating manipulated instances data. Furthermore, these networks extrapolate `hidden' supervised