Tracking the Best Hyperplane with a Simple Budget Perceptron
Giovanni Cavallanti, Nicolò Cesa-Bianchi and Claudio Gentile
Machine Learning Volume 69, Number 2/3, pp. 143-167, 2006.

This is the latest version of this eprint.

Abstract

Shifting bounds for on-line classification algorithms ensure good performance on any sequence of examples that is well predicted by a sequence of smoothly changing classifiers. When proving shifting bounds for kernel-based classifiers, one also faces the problem of storing a number of support vectors that can grow unboundedly, unless an eviction policy is used to keep this number under control. In this paper, we show that shifting and on-line learning on a budget can be combined surprisingly well. First, we introduce and analyze a shifting Perceptron algorithm achieving the best known shifting bounds while using an unlimited budget. Second, we show that by applying to the Perceptron algorithm the simplest possible eviction policy, which discards a random support vector each time a new one comes in, we achieve a shifting bound close to the one we obtained with no budget restrictions. More importantly, we show that our randomized algorithm strikes the optimal trade-off between budget and norm of the largest classifier in the comparison sequence.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

Available Versions of this Item

• Tracking the Best Hyperplane with a Simple Budget Perceptron (deposited 06 October 2006)
  • Tracking the Best Hyperplane with a Simple Budget Perceptron (deposited 14 February 2008) [Currently Displayed]

[Edit]