Modulated Memory Network for Video Object Segmentation

• DOI: 10.3390/math12060863
• Publication Date: 2024-03-15T18:05:25Z
• Authors: Hannan Lu; Zixian Guo; Wangmeng Zuo
• Keywords: matching based; QA1-939; 0202 electrical engineering, electronic engineering, information engineering; modulator; 02 engineering and technology; memory network; video object segmentation; Mathematics

Existing video object segmentation (VOS) methods based on matching techniques commonly employ a reference set comprising historical segmented frames, referred to as ‘memory frames’, facilitate the process. However, these suffer from following limitations: (i) Inherent errors in memory frames can propagate and accumulate when utilized templates for subsequent segmentation. (ii) The non-local technique employed top-leading solutions often fails incorporate positional information, potentially leading incorrect matching. In this paper, we introduce Modulated Memory Network (MMN) VOS. Our MMN enhances matching-based VOS ways: Introducing an Importance Modulator, which adjusts using adaptive weight maps generated confidence associated with each frame. Incorporating Position Modulator that encodes spatial temporal information both current proposed modulator improves accuracy by embedding information. Meanwhile, mitigates error propagation accumulation incorporating confidence-based modulation. Through extensive experimentation, demonstrate effectiveness of our MMN, also achieves promising performance benchmarks.