Striving for Simplicity: The All Convolutional Net

• DOI: 10.48550/arxiv.1412.6806
• Publication Date: 2014-01-01
• Authors: Springenberg, Jos...; Dosovitskiy, Alexey; Brox, Thomas; Riedmiller, Martin
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; 0202 electrical engineering, electronic engineering, information engineering; Computer Science - Neural and Evolutionary Computing; 02 engineering and technology; Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG)

Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We find that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this finding -- and building on other recent work for finding simple network structures -- we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the "deconvolution approach" for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.<br/>accepted to ICLR-2015 workshop track; no changes other than style<br/>