PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

The Forgetron: A Kernel-Based Perceptron on a Fixed Budget
Ofer Dekel, Shai Shalev-Shwartz and Yoram Singer
In: NIPS 2005, December 5, 2005, Vancouver, CA.


The Perceptron algorithm, despite its simplicity, often performs well on online classification tasks. The Perceptron becomes especially effective when it is used in conjunction with kernels. However, a common difficulty encountered when implementing kernel-based online algorithms is the amount of memory required to store the online hypothesis, which may grow unboundedly. In this paper we present and analyze the Forgetron algorithm for kernel-based online learning on a fixed memory budget. To our knowledge, this is the first online learning algorithm which, on one hand, maintains a {\em strict} limit on the number of examples it stores while, on the other hand, entertains a relative mistake bound. In addition to the formal results, we also present experiments with real datasets which underscore the merits of our approach.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2146
Deposited By:Shai Shalev-Shwartz
Deposited On:11 July 2006