Eigen-Distortions of Hierarchical Representations

• DOI: 10.48550/arxiv.1710.02266
• Publication Date: 2017-01-01
• Authors: Alexander Berardino; Valero Laparra; Johannes Ballé; Eero P. Simoncelli
• Keywords: FOS: Computer and information sciences; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology

We develop a method for comparing hierarchical image representations in terms of their ability to explain perceptual sensitivity humans. Specifically, we utilize Fisher information establish model-derived prediction local perturbations an image. For given image, compute the eigenvectors matrix with largest and smallest eigenvalues, corresponding model-predicted most- least-noticeable distortions, respectively. human subjects, then measure amount each distortion that can be reliably detected when added use this test variety mimic sensitivity. find early layers VGG16, deep neural network optimized object recognition, provide better match perception than later layers, 4-stage convolutional (CNN) trained on database ratings distorted quality. On other hand, simple models visual processing, incorporating one or more stages gain control, same ratings, substantially predictions either CNN, any combination VGG16.