Machine Learning Challenges: Evaluating Predictive Uncertainty, Visual Object Classification, and Recognizing Textual Entailment
Joaquin Quinonero Candela, Ido Dagan, Bernardo Magnini and Florence d'Alché-Buc, ed. (2006) Lecture Notes in Artificial Intelligence , Number LNAI 3944. Springer , Heidelberg, Germany . ISBN 3 540 33427 0

Abstract

The first PASCAL Machine Learning Challenges Workshop (MLCW 2005) (see, www.pascal-network.org/Workshops/PC04/) was held in Southampton, UK, during April 11-13, 2005. This conference was organized by the Challenges pro- gramme of the European Network of Excellence PASCAL (Pattern Analysis, Statistical modelling and ComputationAl Learning) in the framework of the IST Programme of the European Community. First annually and now quarterly, the PASCAL Challenges Programme plays the role of selecting and sponsoring chal- lenging tasks, either practical or theoretical. The aim is to raise difficult machine learning questions and to motivate innovative research and development of new approaches. Financial support covers all the work concerning the cleaning and labelling of the data as well as the preparation of evaluation tools for ranking the results. For the first round of the Programme, four challenges were selected according their impact in the machine learning community, supported from sum- mer 2004 to early spring 2005 by PASCAL and finally invited to participate to MLCW 2005 : – the first challenge, called Evaluating Predictive Uncertainty, dealt with the fundamental question of assigning a degree of confidence to the outputs of a classifier or a regressor. – the goal of the second challenge, called Visual Object Classes, was to recog- nise ob jects from a number of visual ob jects classes in realistic scenes. – the third challenge task, called Recognising Textual Entailment, consisted in recognising, given two texts fragments, whether the meaning of one text can be inferred (entailed) from the other. – the fourth challenge was concerned with the assessment of Machine Learning methodologies to extract implicit relations from documents. Each of these challenges raised noticeable attention in the research community, attracting numerous participants. The idea behind having a unique workshop was to make participants to different challenges exchange and benefit from the research experienced in other challenges. For the workshop, the session chairs operated a first selection among submissions leading to 34 oral contributions. This book is concerned with selected proceedings of the three first challenges, providing a large panel of machine learning issues and solutions. A second round of selection was applied to extract the 25 contributed chapters that compose this book, resulting in a selection rate of a half for the three considered challenges whose description follows.

[Edit]