Performance Prediction Challenge
Isabelle Guyon, Amir Saffari, Gideon Dror and Joachim Buhmann
In: WCCI 2006, July 16-21 2006, Vancouver, Canada.
A major challenge for machine learning algorithms
in real world applications is to predict their performance.
We have approached this question by organizing
a challenge in performance prediction for WCCI 2006.
The class of problems addressed are classification problems
encountered in pattern recognition (classification of images,
speech recognition), medical diagnosis, marketing (customer
categorization), text categorization (filtering of spam). Over
100 participants have been trying to build the best possible
classifier from training data and guess their generalization error
on a large unlabeled test set. The challenge scores indicate
that cross-validation yields good results both for model selection
and performance prediction. Alternative model selection
strategies were also sometimes employed with success. The
challenge web site keeps open for post-challenge submissions:
|EPrint Type:||Conference or Workshop Item (Paper)|
|Project Keyword:||Project Keyword UNSPECIFIED|
|Subjects:||Theory & Algorithms|
|Deposited By:||Isabelle Guyon|
|Deposited On:||22 November 2006|