PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Robust Bounds for Classification via Selective Sampling
Nicolò Cesa-Bianchi, Claudio Gentile and Francesco Orabona
In: Proceedings of the 26th International Conference on Machine Learning, Montreal, Canada(2009).

Abstract

We introduce a new algorithm for binary classification in the selective sampling protocol. Our algorithm uses Regularized Least Squares (RLS) as base classifier, and for this reason it can be efficiently run in any RKHS. Unlike previous margin-based semi-supervised algorithms, our sampling condition hinges on a simultaneous upper bound on bias and variance of the RLS estimate under a simple linear label noise model. This fact allows us to prove performance bounds that hold for an arbitrary sequence of instances. In particular, we show that our sampling strategy approximates the margin of the Bayes optimal classifier to any desired accuracy epsilon by asking order of d/epsilon^2 queries (in the RKHS case d is replaced by a suitable spectral quantity). While these are the standard rates in the fully supervised i.i.d. case, the best previously known result in our harder setting was order of d^3/epsilon^4. Preliminary experiments show that some of our algorithms also exhibit a good practical performance.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:5989
Deposited By:Nicolò Cesa-Bianchi
Deposited On:08 March 2010