Training Spiking Neural Networks Using Lessons From Deep Learning

• DOI: 10.48550/arxiv.2109.12894
• Publication Date: 2021-01-01
• Authors: Jason K. Eshraghian; Max Ward; Emre Neftci; Xinxin Wang; Gregor Lenz; Girish Dwivedi; Mohammed Bennamoun; Doo Seok Jeong; Wei Lü
• Keywords: Python; Backpropagation; Neuromorphic engineering

The brain is the perfect place to look for inspiration develop more efficient neural networks. inner workings of our synapses and neurons provide a glimpse at what future deep learning might like. This paper serves as tutorial perspective showing how apply lessons learnt from several decades research in learning, gradient descent, backpropagation neuroscience biologically plausible spiking We also explore delicate interplay between encoding data spikes process; challenges solutions applying gradient-based networks (SNNs); subtle link temporal spike timing dependent plasticity, move towards online learning. Some ideas are well accepted commonly used amongst neuromorphic engineering community, while others presented or justified first time here. fields evolve very rapidly. endeavour treat this document 'dynamic' manuscript that will continue be updated common practices training SNNs change. A series companion interactive tutorials complementary using Python package, snnTorch, made available. See https://snntorch.readthedocs.io/en/latest/tutorials/index.html .