Brain-Inspired Online Adaptation for Remote Sensing with Spiking Neural Network

• DOI: 10.48550/arxiv.2409.02146
• Publication Date: 2024-09-03
• Authors: Dexin Duan; Peilin liu; Wen Fei
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition; Computer Science - Neural and Evolutionary Computing; Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG)

On-device computing, or edge is becoming increasingly important for remote sensing, particularly in applications like deep network-based perception on on-orbit satellites and unmanned aerial vehicles (UAVs). In these scenarios, two brain-like capabilities are crucial sensing models: (1) high energy efficiency, allowing the model to operate devices with limited computing resources, (2) online adaptation, enabling quickly adapt environmental variations, weather changes, sensor drift. This work addresses needs by proposing an adaptation framework based spiking neural networks (SNNs) sensing. Starting a pretrained SNN model, we design efficient, unsupervised algorithm, which adopts approximation of BPTT algorithm only involves forward-in-time computation that significantly reduces computational complexity learning. Besides, propose adaptive activation scaling scheme boost performance, low time-steps. Furthermore, more challenging detection task, confidence-based instance weighting scheme, substantially improves performance task. To our knowledge, this first address SNNs. Extensive experiments seven benchmark datasets across classification, segmentation, tasks demonstrate proposed method outperforms existing domain generalization approaches under varying conditions. The enables energy-efficient fast devices, has much potential such as UAV.