An artificial visual neuron with multiplexed rate and time-to-first-spike coding

• DOI: 10.1038/s41467-024-48103-9
• Publication Date: 2024-05-01T14:48:33Z
• Authors: Fanfan Li; Dingwei Li; Chuanqing Wang; Guolei Liu; Rui Wang; Huihui Ren; Yingjie Tang; Yan Wang; Yitong Chen; Kun Liang; Qi Huang; Mohamad Sawan; Min Qiu; Hong Wang; Bowen Zhu
• Keywords: Predictive coding

Abstract Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial use in spiking neural network (SNN). However, lack multiplexed data coding schemes reduces ability SNN to emulate perception systems. Here, we present an neuron that enables rate temporal fusion (RTF) external information. can code information at different frequencies (rate coding) precise time-to-first-spike (TTFS) coding. This sensory scheme could improve computing capability efficacy neurons. A hardware-based with RTF exhibits good consistency real-world ground truth achieves highly accurate steering speed predictions self-driving vehicles complex conditions. demonstrates feasibility developing efficient spike-based neuromorphic hardware.