PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Performance Prediction Challenge
Isabelle Guyon, Amir Saffari, Gideon Dror and Joachim Buhmann
In: WCCI 2006, July 16-21 2006, Vancouver, Canada.


A major challenge for machine learning algorithms in real world applications is to predict their performance. We have approached this question by organizing a challenge in performance prediction for WCCI 2006. The class of problems addressed are classification problems encountered in pattern recognition (classification of images, speech recognition), medical diagnosis, marketing (customer categorization), text categorization (filtering of spam). Over 100 participants have been trying to build the best possible classifier from training data and guess their generalization error on a large unlabeled test set. The challenge scores indicate that cross-validation yields good results both for model selection and performance prediction. Alternative model selection strategies were also sometimes employed with success. The challenge web site keeps open for post-challenge submissions:

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:2476
Deposited By:Isabelle Guyon
Deposited On:22 November 2006