wiki:2010/spinn10

Using SpiNNaker for Real-World Neural Applications

Members: Andreas Hartel, Adrian Whatley, Alexander Russell, Bernhard Vogginger, Borys Wrobel, Daniel Sonnleithner, David Lester, Thomas Clayton, Emre Neftci, Fabio Stefanini, Daniel B. Fasnacht, Francesco Galluppi, Jorg Conradt, Kevin Brohan, Marc-Olivier Schwartz, Mihai A. Petrovici, Paolo Motto Ros, Johannes Partzsch, Piotr Dudek, Alexander Rast, Rafael Paz-Vicente, Saber Moradi, Johannes Schemmel, Sergio Davies, Simon Friedmann, Sebastian Jeltsch, Tobi Delbruck, André van Schaik


SpiNNaker - A Universal Spiking Neural Network Architecture

SpiNNaker Overview

The SpiNNaker project's architecture DOI-Link mimics the human brain's biological structure and functionality. It offers the possibility of utilising massive parallelism and redundancy to provide resilience in an environment of unreliability and failure of individual components.
The knowledge content and learning ability of the brain is embodied in its evolvable interconnection pattern; this routes a Spike generated by one Neuron to others which are interconnected with it by Axons and these interconnections are modified and extended as a result of the learning and processes.

In SpiNNaker the neural interconnection is controlled by a packet Router within each multi-core processor. Each transmitted packet representing a Spike contains information which identifies its source Neuron; this is used by a multi-core processor's Router to identify whether this packet should be routed to one of its contained Fascicle processors to respond, or should be routed on to one of the six adjacent multi-core processors connected to it as part of the overall SpiNNaker network.

At the beginning of the Attachment Section you will find an essential bibliography regarding the SpiNNaker Project.

For further information please visit http://intranet.cs.man.ac.uk/apt/projects/SpiNNaker/

4 CHIP BOARD



Project Summary

Over the Capo Caccia workshop the group worked to implement and test networks with 2 different neural models: the Izhikevich model and the Leaky Integrate-and-Fire neuron. We created pyNN scripts generating SpiNNaker data file for two example networks, and we tested them against the same models simulated with Brian. Thanks to the Izhikevich Brian script provided by Andre van Schaik we were able to extend Izhikevich neuron support for both pyNN and Brian. The integration of the Izhikevich neuron into pyNN represents the first time this neural type has been thus integrated. Both networks have now run successfully and matched the Brian output. SpiNNaker models use 16-bit representations with 1-ms time resolution; the close fit of these models to Brian output suggests that data-precision concerns arising from the fixed-point notation and semi-coarse time resolution are not significant. Finally we simulated 500 Izhikevich randomly connected neurons in real time. Description and results from the three tests are proposed in the next sections.

There was also a discussion element focussing on the potential integration of cx3d or other evolutionary neural network builders with SpiNNaker. The results are very promising; certainly it seems clear that SpiNNaker can implement the evolutionary algorithms and the neural dynamics with significant acceleration. On-chip implementation of the environment and physics remains an open research area. Using evolutionary algorithms may prove to be a practical method for implementing very large-scale neural networks containing hundreds of millions of neurons on the full-scale 50,000 chip system, for which manual methods are clearly infeasible; fully automated techniques like evolutionary systems will become essential at such large model sizes.

Leaky Integrate-and-Fire

In order to test the implementation of the Leaky Integrate-and-Fire neuron we ran a simulation of a small network, composed of two populations of 10 LIF neurons each, interconnected with a One To One connection (each neuron in Population 1 is connected to the corresponding neuron in Population 2) with fixed weights and delays (see pyNN script attached for details on the network structure).

All neurons in Population 1 receive a constant DC current injection corresponding to a membrane voltage of 10mV, to make them fire 3 times in 500 ms. Results for the stimulation are plotted below (membrane potential of an input neuron).

We implemented the same neuron (same parameters/equation) in a Brian script and we stimulated it with the same current in order to validate the results obtained from SpiNNaker; results show that the neuron dynamics for the SpiNNaker and the Brian script are the same.


Membrane Potential for a neuron in the input population (from SpiNNaker output data)
Input Neuron Membrane Potential - SpiNNaker Data
Result of the Brian comparison script for the LIF network (membrane potential of the input neuron)
Input Neuron Membrane Potential - Brian Data

Izhikevich Model

In order to test the Izhikevich neuron model we implemented a 3 Layer feedforward network composed of 21 Izhikevich neurons (see attached pyNN file for more details):

Layer 1: One Input Neuron is injected with a DC Current corresponding to a 20mV membrane potential (Layer 1) which makes it fire at ~40Hz

Layer 2 and Layer 3: Two Populations of 10 Neurons each

Layer 1 (neuron 0) is connected to all neurons in Layer2 with random delays

Layer 2 is connected to Layer 3 with One To One connections having random weights

We tested single neuron dynamics with a Brian script based on the one provided by Andre van Schaik. Results are qualitatively preserved, despite the high firing frequency and the fixed point implementation of the model on the SpiNNaker. Results below:

Izhikevich Model - Membrane Potential for the input neuron (from SpiNNaker output data)
Input Neuron Membrane Potential - SpiNNaker Data
Result of the Brian comparison script for the Izhikevich network (membrane potential of the input neuron)
Input Neuron Membrane Potential - Brian Data


In order to test the correct propagation and set-up of the random weights and delays we examined the membrane potential for a neuron in Layer 2 and two neurons in Layer 3.
Spikes arriving from the input neuron in Layer 1 to a neuron in Layer 2 make it fire after receiving roughly 4 spikes. Spikes arrive at two neurons in Layer 3 with different time and intensity, according to the parameters for each (single) synaptic connection (SpiNNaker data files describing each connection can be found in the .tar.gz file for the Izhikevich model).
Izhikevoch Model - Membrane Potential for a neuron in Layer 2
Membrane Potential of a neuron in Layer 2
Izhikevich Model - Membrane Potential comparison for two neurons in Layer 3 (incoming random weight/delay connections)
Membrane Potential of two neurons in Layer 3 - random delays and weights


Real time simulation of a randomly connected network composed of 500 Izhikevich neurons

The last model we simulated was a randomly connected network composed of 500 Izhikevich neurons. With this simulation we tested the ability of the SpiNNaker system to run a large simulation in real time, while being able to collect the data needed to generate a raster plot at the end of the simulation.

This network aims to scale down and reproduce the results of the first part of the simulation in Izhikevich's Polychronization paper (delta-frequency oscillations ~2-4 Hz) The network is composed of 500 neurons, divided into excitatory and inhibitory populations in a 4:1 ratio: 400 excitatory Regular Spiking (RS) neurons and 100 Fast Spiking (FS) inhibitory neurons.

Each presynaptic neuron is randomly connected with 30 postsynaptic neurons, with random delays in the range 1-16 ms. Weights are fixed to 10 mV for excitatory connections and -5 mV for inhibitory connections.

The model ran for 1 second simulated/real time, and the raster plot was fetched at the end of the simulation.
500 Neurons Network - Raster Plot

Summary of Results

  • High speed (real time) hardware simulation of heterogeneous neural models
  • Fast switching between neural models, completely transparent to the end user
  • Establishment of a design flow for neural modeling on hardware using a standard pyNN interface
  • Original implementation of the Izhikevich neuron in pyNN
  • Exploration of the possibility to use the SpiNNaker hardware in evolving neural networks

Essential Bibliography

Last modified 9 years ago Last modified on 05/26/10 01:56:47

Attachments (23)