jAER: Event-based processing algorithms

Members: Adrian Whatley, Jonathan Tapson, Kaijun Yi, Rainer Beutelmann, Shih-Chii Liu, Tobi Delbruck, Tao Zhou, Xavier Hinaut

The open source project  jAER (java AER, pronounced jay-er) enables application of neuromorphic AER sensors and actuators by binding them with the power of PCs. This tutorial will teach how to use in connection with the spike event-based  dynamic vision sensor (DVS) silicon retina and  AER-EAR silicon cochlea.

We'll present jAER in two sessions and then leave the rest of the learning to the projects where it might be used. See the attachments below for downloads of jAER, the Java Development Kit, and the netbeans IDE.

Available hardware

• 12x DVS128 silicon retinas with high speed USB interfaces (USB bus powered)
• 1x DVS128 silicon retinas with parallel AER interfaces (USB bus powered)
• Variety of CS-mount lenses
• Tripods
• USB cables and hubs
• A few fast digital servos
• A few servo controllers interfaces, each of which can control 4 servos, with USB bus-powered interfaces and supporting classes in jAER

Preparation

1. Sign up for the subversion tutorial? if you are not experienced using subversion.

The URL you need to use in subversion to browse or check out the jAER repository is  https://jaer.svn.sourceforge.net/svnroot/jaer/trunk .

1. Read the attached paper attachment:delbruckeventbasedvision_gcoe_symp_2008.pdf for background.

If you will be doing a project using jAER,

1. The full checkout of jAER from sourceforge takes a long time at the workshop because it is about 90MB and takes about 30 minutes at off-peak times. See the attachments below for a recent working copy.
2. Unzip the zip somewhere. You now have a subversion "working copy", but it may not be up to date.
3. Install a subversion client if you are running windows (TortoiseSVN) or mac OS (Versions). TortoiseSVN installers are attached below. Tobi has some Versions licenses.
4. You now should be able to quickly update your working copy. Do a subversion update on your working copy of jAER.
5. You should be able to run jAER from the root of this extract. Try to run jAERViewer.exe (under windows) or jAERViewer.sh from linux or Mac OS X. If you get an error like Class not found, you may also need to install or update your java runtime; jAER needs at least java 1.6.
6. Under Windows or linux, you can get the JDK (Java development kit) and the netbeans IDE (integrated development environment) by installing the jdk-netbeans bundle attached below. For a different platform, go to  http://netbeans.org for netbeans installers. Choose the Jave SE (standard edition) download.

If you have trouble at some point, please ask 'Tobi' for help. Other participants also have experience.

Troubleshooting

• Under mac OS, the default choice for the JRE (Java runtime environment) is Java 1.6; you need to change the platform used in netbeans. Ask Tobi Delbruck for help here.
• JOGL: The Java Open GL libraries used in jAER are part of the jAER checkout. But some of the native code JOGL modules for other platforms may not be up to date. We may need to refresh these to the latest JOGL. In particular, at the moment we have only 32 bit linux jogl runtimes. You should use a 32-bit JDK/JRE on your linux, even if you run 64 bit linux.
• See  http://jaer.wiki.sourceforge.net/jAER+installation for more tips about installing jAER. Do not download the jAER package from sourceforge - we are using subversion working copies for development.
• The most frequent problem is that the native libraries such as those associated with JOGL are not found. The folder where the .dll's (windows), .so's (linux), .jnilib's (macosx) should be listed in order in the JVM environment variable java.library.path. There should only be one definition of java.library.path with all folders in sequence separated by the (system dependent) path separator: ";" (semicolon) under windows, ":" (colon) under linux. For example, under windows:

  -Djava.library.path=jars;jars\SiLabsNativeWindows
  

• The above definition assumes the startup folder is java, not the root of jaer. The path to jars is then relative to the java startup folder.

Path and file separators

OS	Path separator	File separator
Windows	; (semicolon)	\ (backslash
MacOSX	: (colon)	/ (fwdslash
linux	: (colon)	/ (fwdslash

Background information

1. Read the paper Frame-free dynamic digital vision to see how spikes are processing in jAER from the  DVS silicon retina.
2. Have a look at the  jAER wiki on SourceForge? which has lots of information about jAER.
3. Sign up for a free account at SourceForge using  Create a SourceForge.net Account and send Tobi Delbruck your SF unix username. This username will be needed to add your as a jAER developer so that you can commit your code.

Using matlab

[ As described on the jAER wiki], you can read logged event data into matlab or generate events in matlab to process with jAER.

You need to add the folder host/matlab from the jAER root to your matlab search path to use the loadaerdat.m and saveaerdat.m functions in matlab. These functions load and save raw address/timestamp data. To extract the actual pixel addresses you can use the functions in the various subfolders.

jAER devloper setup for other than netbeans

(For experts). Your jAER launcher needs to include something like the following set of switches to set up the JNI path and other options. These are normally part of your IDE's project options for launching classes.

-Djava.util.logging.config.file=conf/Logging.properties -Dsun.java2d.opengl=false  
-Dsun.java2d.noddraw=true 
-Djava.library.path=jars;jars\SiLabsNativeWindows
-Djava.security.policy=applet.policy