PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Model selection by bootstrap penalization for classification
Magalie Fromont
In: COLT 2004, Juillet 2004, Banff,Canada.

Abstract

We consider the binary classification problem. Given an i.i.d. sample drawn from the distribution of an X x {0,1}-valued random pair, we propose to estimate the so-called Bayes classifier by minimizing the sum of the empirical classification error and a penalty term based on Efron's or i.i.d. weighted bootstrap samples of the data. We obtain exponential inequalities for such bootstrap type penalties, which allow us to derive non-asymptotic properties for the corresponding estimators. In particular, we prove that these estimators achieve the global minimax risk over sets of functions built from Vapnik-Chervonenkis classes. The obtained results generalize Koltchinskii's and Bartlett, Boucheron, Lugosi's ones for Rademacher penalties that can thus be seen as special examples of bootstrap type penalties.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
ID Code:716
Deposited By:Michele Sebag
Deposited On:30 December 2004