Model selection by bootstrap penalization for classification ## AbstractWe consider the binary classification problem. Given an i.i.d. sample drawn from the distribution of an X*{0,1}-valued random pair, we propose to estimate the so-called Bayes classifier by minimizing the sum of the empirical classification error and a penalty term based on Efron's or i.i.d. weighted bootstrap samples of the data. We obtain exponential inequalities for such bootstrap type penalties, which allow us to derive non-asymptotic properties for the corresponding estimators. In particular, we prove that these estimators achieve the global minimax risk over sets of functions built from Vapnik-Chervonenkis classes. The obtained results generalize Koltchinskii (2001) and Bartlett, Boucheron, Lugosi's (2002) ones for Rademacher penalties that can thus be seen as special examples of bootstrap type penalties.
[Edit] |