Depthwise Separable Convolutions Allow for Fast and Memory-Efficient Spectral Normalization

• DOI: 10.48550/arxiv.2102.06496
• Publication Date: 2021-01-01
• Authors: Christina Runkel; Christian Etmann; Michael Møller; Carola‐Bibiane Sc...
• Keywords: Normalization

An increasing number of models require the control spectral norm convolutional layers a neural network. While there is an abundance methods for estimating and enforcing upper bounds on those during training, they are typically costly in either memory or time. In this work, we introduce very simple method normalization depthwise separable convolutions, which introduces negligible computational overhead. We demonstrate effectiveness our image classification tasks using standard architectures like MobileNetV2.