2011/riobva11

Robotic Implementation of Object-Based Visual Attention

Results

Introduction

Fig. 1: Image of the system setup. The robot consisted of a camera, mounted on a pan-tilt unit.

The robotic eye makes a series of saccades to different objects across the visual scene: the foveated object is the attentional target, and a number of unattended objects are also present in the perifovea. A reward can be given for a saccade to the red, the white or neither object, but colour is not used as a feature in the object recognition process.

Experiment 1: Offline Learning

Fig. 2: Histogram characterising the likelihood of making a saccade to a target cue. Y-axis, number of hits over 100 trials; 5 trials were used to create this graph. Results for the case where the triangle is rewarding are shown in the upper panel.

Fig. 3: Cumulative number of saccades (out of 100) to each category during the 3 different reward conditions.

Experiment 2: Online Learning

In the second experiment, the system learned visual features online. Features took the form of a 6 x 6 pixel patch of the image luminance.

Fig. 4:

Link to videos

Illustrative Example of SOM at work:

Meetings

Issues from Meeting 03-05-11

  • Interaction between feature maps and retinotopic maps
  • Nature of features
  • Implementation of the number of features used to identify an object
  • Kevin to send papers and an example of the SOM

Issues from Meeting 04-05-11

  • Continuation of implementation (Selection of attentional target, now calibrating the camera to initiate saccade)
  • SOM is now working in APRON
  • Pan-tilt controller working

Issues from Meeting 06-05-11

  • Implementation is complete, but poor performance.
  • Work to understand if this is a bug or an algorithmic problem
  • Issues considered: size of the matching templates, variation in positions of the training templates, interference between similar features, poor representation on the SOM

Issues from Meeting 9-05-11

  • Poor performance persists despite numerous attempts to improve
  • Discussion of algorithm
  • Kevin to gather large feature set for Jan

Issues from Meeting 10-05-11

  • Nice results in the offline version
  • Discussion of algorithm for SOM
  • Tao will try to generate sparse feature set. Kevin to gather 500 images for this.

Attachments