Training Deep Spiking Neural Networks Using Backpropagation
Top Cited Papers
Open Access
- 8 November 2016
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Neuroscience
- Vol. 10, 508
- https://doi.org/10.3389/fnins.2016.00508
Abstract
Deep spiking neural networks (SNNs) hold the potential for improving the latency and energy efficiency of deep neural networks through data-driven event-based computation. However, training such networks is difficult due to the non-differentiable nature of spike events. In this paper, we introduce a novel technique, which treats the membrane potentials of spiking neurons as differentiable signals, where discontinuities at spike times are considered as noise. This enables an error backpropagation mechanism for deep SNNs that follows the same principles as in conventional deep networks, but works directly on spike signals and membrane potentials. Compared with previous methods relying on indirect training and conversion, our technique has the potential to capture the statistics of spikes more precisely. We evaluate the proposed framework on artificially generated events from the original MNIST handwritten digit benchmark, and also on the N-MNIST benchmark recorded with an event-based dynamic vision sensor, in which the proposed method reduces the error rate by a factor of more than three compared to the best previous SNN, and also achieves a higher accuracy than a conventional convolutional neural network (CNN) trained and tested on the same data. We demonstrate in the context of the MNIST task that thanks to their event-driven operation, deep SNNs (both fully connected and convolutional) trained with our method achieve accuracy equivalent with conventional neural networks. In the N-MNIST example, equivalent accuracy is achieved with about five times fewer computational operations.Keywords
This publication has 31 references indexed in Scilit:
- A million spiking-neuron integrated circuit with a scalable communication network and interfaceScience, 2014
- Improved Reconstruction of 4D-MR Images by Motion PredictionsLecture Notes in Computer Science, 2014
- Event-driven contrastive divergence for spiking neuromorphic systemsFrontiers in Neuroscience, 2014
- Real-time classification and sensor fusion with a spiking deep belief networkFrontiers in Neuroscience, 2013
- Computation with Spikes in a Winner-Take-All NetworkNeural Computation, 2009
- Sparse Coding via Thresholding and Local Competition in Neural CircuitsNeural Computation, 2008
- A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision SensorIEEE Journal of Solid-State Circuits, 2008
- Unsupervised Learning of Visual Features through Spike Timing Dependent PlasticityPLoS Computational Biology, 2007
- On the computational power of circuits of spiking neuronsJournal of Computer and System Sciences, 2004
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998