CISA: Context Substitution for Image Semantics Augmentation

• DOI: 10.3390/math11081818
• Publication Date: 2023-04-12T05:35:08Z
• Authors: Sergey Nesteruk; Ilya Zherebtsov; Svetlana Illarionova; Dmitrii Shadrin; Andrey Somov; Sergey V. Bezzateev; Tatiana Yelina; Vladimir Denisenko; Ivan Oseledets
• Keywords: Substitution (logic)

Large datasets catalyze the rapid expansion of deep learning and computer vision. At same time, in many domains, there is a lack training data, which may become an obstacle for practical application vision models. To overcome this problem, it popular to apply image augmentation. When dataset contains instance segmentation masks, possible instance-level It operates by cutting from original pasting new backgrounds. This article challenges with objects present various domains. We introduce Context Substitution Image Semantics Augmentation framework (CISA), focused on choosing good background images. compare several ways find backgrounds that match context test set, including Contrastive Language–Image Pre-Training (CLIP) retrieval diffusion generation. prove our augmentation method effective classification, segmentation, object detection different complexity model types. The average percentage increase accuracy across all tasks fruits vegetables recognition 4.95%. Moreover, we show Fréchet Inception Distance (FID) metrics has strong correlation accuracy, can help choose better without training. negative between FID augmented 0.55 experiments.