Finding Visual Saliency in Continuous Spike Stream

FOS: Computer and information sciences Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition
DOI: 10.1609/aaai.v38i7.28610 Publication Date: 2024-03-25T10:02:36Z
ABSTRACT
As a bio-inspired vision sensor, the spike camera emulates operational principles of fovea, compact retinal region, by employing discharges to encode accumulation per-pixel luminance intensity. Leveraging its high temporal resolution and neuromorphic design, holds significant promise for advancing computer applications. Saliency detection mimic behavior human beings capture most salient region from scenes. In this paper, we investigate visual saliency in continuous stream first time. To effectively process binary stream, propose Recurrent Spiking Transformer (RST) framework, which is based on full spiking neural network. Our framework enables extraction spatio-temporal features while maintaining low power consumption. facilitate training validation our proposed model, build comprehensive real-world spike-based dataset, enriched with numerous light conditions. Extensive experiments demonstrate superior performance comparison other network-based methods. exhibits substantial margin improvement capturing highlighting not only provides new perspective segmentation but also shows paradigm SNN-based transformer models. The code dataset are available at https://github.com/BIT-Vision/SVS.
SUPPLEMENTAL MATERIAL
Coming soon ....
REFERENCES (0)
CITATIONS (0)
EXTERNAL LINKS
PlumX Metrics
RECOMMENDATIONS
FAIR ASSESSMENT
Coming soon ....
JUPYTER LAB
Coming soon ....