SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption

• DOI: 10.48550/arxiv.2106.15147
• Publication Date: 2021-01-01
• Authors: Dara Bahri; Heinrich Jiang; Yi Tay; Donald Metzler
• Keywords: Leverage (statistics); Benchmark (surveying); Feature Learning; Representation; Feature (linguistics); Labeled data; Supervised Learning

Self-supervised contrastive representation learning has proved incredibly successful in the vision and natural language domains, enabling state-of-the-art performance with orders of magnitude less labeled data. However, such methods are domain-specific little been done to leverage this technique on real-world tabular datasets. We propose SCARF, a simple, widely-applicable for learning, where views formed by corrupting random subset features. When applied pre-train deep neural networks 69 real-world, classification datasets from OpenML-CC18 benchmark, SCARF not only improves accuracy fully-supervised setting but does so also presence label noise semi-supervised fraction available training data is labeled. show that complements existing strategies outperforms alternatives like autoencoders. conduct comprehensive ablations, detailing importance range factors.