Diverse Imagenet Models Transfer Better

• DOI: 10.48550/arxiv.2204.09134
• Publication Date: 2022-01-01
• Authors: Niv Nayman; Avram Golbert; Asaf Noy; Ping Tan; Lihi Zelnik‐Manor
• Keywords: Transferability; Transfer of learning; Supervised Learning

A commonly accepted hypothesis is that models with higher accuracy on Imagenet perform better other downstream tasks, leading to much research dedicated optimizing accuracy. Recently this has been challenged by evidence showing self-supervised transfer than their supervised counterparts, despite inferior This calls for identifying the additional factors, top of accuracy, make transferable. In work we show high diversity features learnt model promotes transferability jointly Encouraged recent results models, propose a method combines and pretraining generate both as result transferability. We demonstrate our several architectures multiple including single-label multi-label classification.