BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks

• DOI: 10.48550/arxiv.2110.14092
• Publication Date: 2021-01-01
• Authors: Guangzhi Tang; N.M Saravana Kumar; Ioannis Polykretis; Konstantinos P. M...
• Keywords: MNIST database; Neuromorphic engineering; Deep Neural Networks

Spiking neural networks (SNN) are delivering energy-efficient, massively parallel, and low-latency solutions to AI problems, facilitated by the emerging neuromorphic chips. To harness these computational benefits, SNN need be trained learning algorithms that adhere brain-inspired principles, namely event-based, local, online computations. Yet, state-of-the-art training based on backprop does not follow above principles. Due its limited biological plausibility, application of requires non-local feedback pathways for transmitting continuous-valued errors, relies gradients from future timesteps. The introduction biologically plausible modifications has helped overcome several limitations, but limits degree which is approximated, hinders performance. We propose a gradient-based algorithm functionally equivalent backprop, while adhering all three introduced multi-compartment spiking neurons with local eligibility traces compute required learning, periodic "sleep" phase further improve approximation during Hebbian rule aligns feedforward weights. Our method achieved same level performance as multi-layer fully connected MNIST (98.13%) event-based N-MNIST (97.59%) datasets. deployed our Intel's Loihi train 1-hidden-layer network MNIST, obtained 93.32% test accuracy consuming 400 times less energy per sample than BioGrad GPU. work shows optimal feasible in computing, pursuing plausibility can better capture benefits this computing paradigm.