PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Discussion of V.Koltchinskii's 2004 IMS Medallion Lecture paper, "Local Rademacher complexities and oracle inequalities in risk minimization"
Gilles Blanchard and Pascal Massart
Annals of Statistics Volume 34, Number 6, 2006.


Discussion paper for V. Koltchinskii's paper in the Annals of Statistics. We discuss the current state of model selection and the gap between theory and practice. We show that the Hold-out procedure is adaptive to Tsybakov's noise condition and this fact has an elementary proof. Therefore, theoretical results for model selection face a double adversary: from a theoretical point of view, Hold-out is a simple and efficient challenger to more elaborate methods. From a practical point of view, the problem is that strongly data-dependent penalization schemes such as cross-validation do not enter in the framework of the current theory. We conclude by proposing some possible promising directions for dealing with data-driven penalties that are both ready to be used in practice and theoretically efficient.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2520
Deposited By:Gilles Blanchard
Deposited On:22 November 2006