PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Step Size-Adapted Online Support Vector Learning
Alexandros Karatzoglou, S V N Vishwanathan, Nicol N. Schraudolph and Alex Smola
In: ISSPA 2005, 28-31 Aug 2005, Sydney, Australia.

Abstract

We present an online Support Vector Machine (SVM) that uses Stochastic Meta-Descent (SMD) to adapt its step size automatically. We formulate the online learning problem as a stochastic gradient descent in Reproducing Kernel Hilbert Space (RKHS) and translate SMD to the nonparametric setting, where its gradient trace parameter is no longer a coefficient vector but an element of the RKHS. We derive efficient updates that allow us to perform the step size adaptation in linear time. We apply the online SVM framework to a variety of loss functions and in particular show how to achieve efficient online multiclass classification. Experimental evidence suggests that our algorithm outperforms existing methods.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Poster)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:1672
Deposited By:Nicol Schraudolph
Deposited On:28 November 2005