PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Tracking the Best Hyperplane with a Simple Budget Perceptron
Nicolò Cesa-Bianchi and Claudio Gentile
In: 19th Annual Conference on Learning Theory, 22-25 Jun 2006, Pittsburgh, USA.

There is a more recent version of this eprint available. Click here to view it.

Abstract

Shifting bounds for on-line classification algorithms ensure good performance on any sequence of examples that is well predicted by a sequence of smoothly changing classifiers. When proving shifting bounds for kernel-based classifiers, one also faces the problem of storing a number of support vectors that can grow unboundedly, unless an eviction policy is used to keep this number under control. In this paper, we show that shifting and on-line learning on a budget can be combined surprisingly well. First, we introduce and analyze a shifting Perceptron algorithm achieving the best known shifting bounds while using an unlimited budget. Second, we show that by applying to the Perceptron algorithm the simplest possible eviction policy, which discards a random support vector each time a new one comes in, we achieve a shifting bound close to the one we obtained with no budget restrictions. More importantly, we show that our randomized algorithm strikes the optimal trade-off between budget and norm of the largest classifier in the comparison sequence.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:2232
Deposited By:Nicolò Cesa-Bianchi
Deposited On:06 October 2006

Available Versions of this Item