Ensemble methods for classification and regression


Ensemble or boosting techniques are a class of meta-algorithms that strategically combine multiple simple base models in order to generate an aggregated model, displaying superior generalization performance. One of the most popular boosting procedures is Adaboost (Freund and Schapire, 1996), which is ubiquitously applied in machine learning, statistics, and even in common everyday use as the most widespread face detection algorithm for digital cameras (Viola and Jones, 2004). Surprisingly, the theoretical reasons and conditions for the success of boosting methods have only been partially elucidated. Leo Breiman for instance recently conjectured that a crucial element for such methods to work, is that the base models that are being aggregated must be sufficiently variable. In fact, Breiman (2001) supported this insight with a new ensemble method called "Random forest", that explicitly introduces variability among the base models and often reaches performances comparable to Adaboost, despite being much simpler. What is most appealing to Breiman's discovery is the demonstration that ensemble methods can be non-adaptive, and aggregate base models that are trained independently, and therefore potentially in parallel (Adaboost on the other hand requires a sequential training of the base models). The second appealing aspect is that noise, in the form of variability among elements, is a crucial ingredient of the algorithm.


The scope of the proposed workgroup is to explore whether these ideas can be exploited in neuromorphic devices. The central question will be whether device mismatch and operating noise could be utilized as the sources of variability needed in ensemble methods. Part of the work will consist in modeling and characterizing relevant forms of noise in current neuromorphic setups, and try to understand how it can instantiate the necessary variability needed by ensemble methods. We will then simulate these devices on standard classification and regression problems, and envisage applications in devices that are available at the workshop.


Please log in to see this part


Last modified 5 years ago Last modified on 06/11/14 10:25:11

Attachments (16)