One objective of Spiking Neural Networks is a very efficient computation in terms of energy consumption. To achieve this target, a small spike rate is of course very beneficial since the event-driven nature of such a computation. However, as the network becomes deeper, the spike rate tends to increase without any improvements in the final results. On the other hand, the introduction of a penalty on the excess of spikes can often lead the network to a configuration where many neurons are silent, resulting in a drop of the computational efficacy. In this paper, we propose a learning strategy that keeps the spike rate under control, by (i) changing the loss function to penalize the spikes generated by neurons after the first ones, and by (ii) proposing a two-phase training that avoids silent neurons during the training.

Reducing the Spike Rate in Deep Spiking Neural Networks

Fontanini, Riccardo
;
Esseni, David;Loghi, Mirko
2022-01-01

Abstract

One objective of Spiking Neural Networks is a very efficient computation in terms of energy consumption. To achieve this target, a small spike rate is of course very beneficial since the event-driven nature of such a computation. However, as the network becomes deeper, the spike rate tends to increase without any improvements in the final results. On the other hand, the introduction of a penalty on the excess of spikes can often lead the network to a configuration where many neurons are silent, resulting in a drop of the computational efficacy. In this paper, we propose a learning strategy that keeps the spike rate under control, by (i) changing the loss function to penalize the spikes generated by neurons after the first ones, and by (ii) proposing a two-phase training that avoids silent neurons during the training.
2022
9781450397896
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11390/1231911
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? ND
social impact