PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Result Analysis of the NIPS 2003 Feature Selection Challenge
Isabelle Guyon, Steve Gunn, Asa Ben-Hur and Gideon Dror
In: NIPS 2004, 13-16 Dec 2004, Vancouver, Canada.

Abstract

The NIPS 2003 workshops included a feature selection competition organized by the authors. We provided participants with five datasets from different application domains and called for classification results using a minimal number of features. The competition took place over a period of 13 weeks and attracted 78 research groups. Participants were asked to make on-line submissions on the validation and test sets, with performance on the validation set being presented immediately to the participant and performance on the test set presented to the participants at the workshop. In total 1863 entries were made on the validation sets during the development period and 135 entries on all test sets for the final competition. The winners used a combination of Bayesian neural networks with ARD priors and Dirichlet diffusion trees. Other top entries used a variety of methods for feature selection, which combined filters and/or wrapper or embedded methods using Random Forests, kernel methods, or neural networks as a classification engine. The results of the benchmark (including the predictions made by the participants and the features they selected) and the scoring software are publicly available. The benchmark is available at http://www.nipsfsc.ecs.soton.ac.uk/ for post-challenge submissions to stimulate further research.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:848
Deposited By:Steve Gunn
Deposited On:01 January 2005