InsPro: Propagating Instance Query and Proposal for Online Video Instance Segmentation

• DOI: 10.48550/arxiv.2301.01882
• Publication Date: 2023-01-01
• Authors: Fei He; Haoyang Zhang; Naiyu Gao; Jian Jia; Yanhu Shan; Xin Zhao; Kaiqi Huang
• Keywords: Association (psychology)

Video instance segmentation (VIS) aims at segmenting and tracking objects in videos. Prior methods typically generate frame-level or clip-level object instances first then associate them by either additional heads complex matching algorithms. This explicit association approach increases system complexity fails to fully exploit temporal cues In this paper, we design a simple, fast yet effective query-based framework for online VIS. Relying on an query proposal propagation mechanism with several specially developed components, can perform accurate implicitly. Specifically, based set of query-proposal pairs propagated from previous frames. pair is learned bind one specific across frames through conscientiously strategies. When using such predict the current frame, not only generated automatically associated its precursors frames, but model gets good prior predicting same object. way, naturally achieve implicit parallel elegantly take advantage clues To show effectiveness our method InsPro, evaluate it two popular VIS benchmarks, i.e., YouTube-VIS 2019 2021. Without bells-and-whistles, InsPro ResNet-50 backbone achieves 43.2 AP 37.6 these benchmarks respectively, outperforming all other methods.