Finding Visual Saliency in Continuous Spike Stream

• DOI: 10.1609/aaai.v38i7.28610
• Publication Date: 2024-03-25T10:02:36Z
• Authors: Lin Zhu; Xianzhang Chen; Xiao Wang; Hua Huang
• Keywords: FOS: Computer and information sciences; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition

As a bio-inspired vision sensor, the spike camera emulates operational principles of fovea, compact retinal region, by employing discharges to encode accumulation per-pixel luminance intensity. Leveraging its high temporal resolution and neuromorphic design, holds significant promise for advancing computer applications. Saliency detection mimic behavior human beings capture most salient region from scenes. In this paper, we investigate visual saliency in continuous stream first time. To effectively process binary stream, propose Recurrent Spiking Transformer (RST) framework, which is based on full spiking neural network. Our framework enables extraction spatio-temporal features while maintaining low power consumption. facilitate training validation our proposed model, build comprehensive real-world spike-based dataset, enriched with numerous light conditions. Extensive experiments demonstrate superior performance comparison other network-based methods. exhibits substantial margin improvement capturing highlighting not only provides new perspective segmentation but also shows paradigm SNN-based transformer models. The code dataset are available at https://github.com/BIT-Vision/SVS.