PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Tracking the Best Hyperplane with a Simple Budget Perceptron
Giovanni Cavallanti, Nicolò Cesa-Bianchi and Claudio Gentile
Machine Learning Volume 69, Number 2/3, pp. 143-167, 2006.

This is the latest version of this eprint.

Abstract

Shifting bounds for on-line classification algorithms ensure good performance on any sequence of examples that is well predicted by a sequence of smoothly changing classifiers. When proving shifting bounds for kernel-based classifiers, one also faces the problem of storing a number of support vectors that can grow unboundedly, unless an eviction policy is used to keep this number under control. In this paper, we show that shifting and on-line learning on a budget can be combined surprisingly well. First, we introduce and analyze a shifting Perceptron algorithm achieving the best known shifting bounds while using an unlimited budget. Second, we show that by applying to the Perceptron algorithm the simplest possible eviction policy, which discards a random support vector each time a new one comes in, we achieve a shifting bound close to the one we obtained with no budget restrictions. More importantly, we show that our randomized algorithm strikes the optimal trade-off between budget and norm of the largest classifier in the comparison sequence.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:3699
Deposited By:Nicolò Cesa-Bianchi
Deposited On:14 February 2008

Available Versions of this Item