This workgroup is for who wants to start working with neuromorphic VLSI hardware for doing spike-based computation and learning. All groups using INI VLSI hardware converge here for the first couple of meetings.

Introduction

Over the last couple of decades the research of neuromorphic engineering community has focused on the understanding of low-level sensory processing and systems infrastructure. It is time to apply this knowledge and infrastructure to addressing higher-level problems in perception, cognition, and learning.

The goal of this workgroup is to explore the possibilities of neuromorphic hardware with learning capabilities implemented in VLSI to solve traditional classification tasks such as character recognition, speech recognition and so on. We will ship a hardware setup from Zurich, with multi-neuron chips comprising arrays of integrate-and-fire neurons with dynamic synapses and spike-based learning algorithm for synaptic modifications. Participants will use general-purpose software tools we implemented as Python modules for an easy design and realization of experiments of spiking neural-networks using this hardware.

If you have interests in neuromorphic hardware and on how to use this hardware for building general purpose real-time cognitive systems this is the right place to start. Previous experience not required.

Relevant Information

The neuromorphic setup of the INI NCS group

The Institute of Neuroinformatics (INI) in Zurich developed a hardware and software setup of neuromorphic chips implemented in VLSI technology communicating through Address Event Representation protocol (AER). Each chip comprises arrays of Leaky-Integrate-And-Fire neurons with dynamic synapses. A spike-based learning algorithm for synaptic weights modification is also implemented, as well as short-term depression and winner-take-all connectivity.

During the last two years the NCS group has put much effort in developing a complete package of Python modules to design, realize and analyze experiments using spiking neural-network in hardware. The pyNCS software provides tools for the creation of neural-networks with arbitrary topologies and automatic tuning of hardware parameters. Real-time interaction and monitoring of the activity of the network is possible. If you have a background in using software simulators of spiking neural-networks the transition to hardware should be fairly smooth (and hopefully exciting!). More details about the hardware will be provided during the first week.

Stochastic, spike-based learning algorithm

Plastic synapses on the chips are modified by a spike-based learning algorithm. Pre-synaptic spikes induce plasticity which depends on the distribution of the post-synaptic membrane potential through time. Check out the Fusi_etal00 and Brader_etal07 papers attached below.

Projects

In principle, any classification task can be implemented in hardware. The basic experiment consists in choosing the type of stimuli we decide to try classification on. They can be visual, acoustic or everything else that can be translated into trains of spikes. Stimuli can be provided by neuromorphic sensors for real-time classification or offline by software stimulation via USB and a client computer. We try first by providing offline stimuli from standard database sets but we could also try recording new stimuli we want to classify.

After the first days of tutorials we are going to go over some ideas about specific contexts (character recognition, word segmentation, ...). Once the basic experiment is realized and the goal is reached it is fairly easy to extend the exercise to other contexts.

Put ideas here...

Hand-written digits classification

The MNIST database is a benchmark database used in traditional computer-science approaches for classification algorithms. This type of classification has never been achieved using spike-based neurmorphic hardware in VSLI.

The basic idea is to convert the 28x28=784 pxl images of the database into 784 trains of spike. Each train is a Poisson spike-train with mean frequency proportional to the intensity of the corresponding pixel. The spiking stimuli can be then classified by an hardware based perceptron. The chip we will use for learning has 128 neurons with 28 plastic, 2 excitatory and 2 inhibitory synapses each. Synapses can be merged to have 64 neurons with double the number of synapses, 32 with 3x synapses, and so on.

Even though we already know this is not going to compete with the state-of-the-art classification algorithms of traditional software-based approach, it represents a milestone for a hardware-based system. Data pre-processing, population coding and other tricks can be implemented in future versions in order to improve performances.

Spoken-digits classification

Same technique as in hand-written digits classification. Input stimuli are cochleograms or spectrograms from spoken-digits.

Documentation

All the documentation is nicely formatted on a  CapoCaccia local server.

Prerequisites

Hardware requirements

We will provide all neuromorphic hardware you'll need.

Since the whole neuromorphic setup runs through a TCP server-client architecture you can use your own laptop to perform experiments! Check out the Software Requirements section.

Software requirements

The software has been tested on *nix machines (Linux). If you have Windows you need to install a VirtualMachine (Google for "Virtual Box" or "Windows Virtual PC") with Linux or use our live USB system we will provide at the workshop. If you are going to perform a fresh-new linux installation, we suggest Ubuntu or Fedora distributions.

Python is installed on maybe every linux distribution, for sure on Ubuntu and Fedora. Our software works under python 2.6 but should also work on every more recent version, such as 2.7 and 3.0. Make sure your python installation is working by typing python in a terminal. If you don't have errors you are now inside the python shell. Type CTRL+d twice to exit.

Standard packages you require to install

python, python-dev, python-numpy, python-scipy, pydot, python-matplotlib

Optional: enthought-chaco (for realtime visualization of spike activity)

Custom Python packages can be found in attachment. Download and uncompress ini_vlsi_software_complete.tar.gz:

$> tar zxvf ini_vlsi_software_complete.tar.gz

Download and install

pyNCS

python package for spike-based neural networks on hardware.

Dependencies

pydot, pyST, pyAex, NeuroTools_NCS

Optional

pyTune

automatic parameter calibration

pyMap

mapping-table gui for pyNCS

mdp

a modular data-processing python-module

pyAMDA

in case you need to control hardware as a server on your laptop

brian

spiking neural-networks software-simulator

How to install.

$> cd package/you/want/to/install
$> sudo python setup.py install

If you have Red Hat based distribution, like Fedora, you can install the packages by first becoming super-user:

$> cd package/you/want/to/install
$> su
Password:
$> python setup.py install

To check that you have properly installed the required packages you can start python and import the package:

$> python
   
   >>> import pyNCS

If you get no errors the installation was successful. Try the same with all the required packages.

Requirements

Basic knowledge about spiking neural networks is suggested. Some experience in classification experiments is a plus (train-validation-test, etc.). Basic programming skills are suggested (Python).

Tutorial

Now you are ready to take a look at an incredibly detailed step-by-step tutorial to start with the hardware.

Where to find us

Our base location is at the bottom of the disco, right below the stage.

An almost-real-time log

Meeting 1: overview and ideas

Tue. 3rd, 15.30, disco

• The software architecture: pyNCS
• The hardware architecture
• Possible projects
• Next meeting: Wed. 4th, 15:30-17:00, @disco

Meeting 2: hands-on step-by-step

Many interested people started installing linux operating systems on virtual machines. This has revealed to be the most efficient solution because of the many "satellite" packages that need to be installed on MacOsX machines. Windows is not supported at all. Only few people had a 4Gb usb stick so we needed to buy some. Next time we should have around 4 usb sticks with operating system to install and also some live distribution on usb stick for people that don't have a virtual machine option.

Software development

Software is always under development. Our packages (pyNCS, pyAMDA, pyST, pyTune, ...) are stable and useful but there is still room for improvements. Having people from different backgrounds as first-time users of our packages is an incredible chance to find-out what to change to make pyNCS and sons really ready for distribution to the community. We strongly encourage participants to give us feedback from their experiences with our software. Starting from the installation... :)

Many people offered to report feedback about the documentation which in general was successful in helping poeple to for their first steps with pyNCS and Co. Nevertheless the participants needed a constant help from the workgroup team and this could probably solved by describing the different option that one has for example for connections (all2all, etc.)

An application for real-time hardware classification

What could be a real application for a real-time classification system in VLSI hardware?

References

See attachments (click on the download icon)

DON'T RIGHT CLICK ON THE ATTACHMENTS INSTEAD CLICK ON THE LITTLE ICON