PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A second-order Perceptron algorithm
Nicolò Cesa-Bianchi, Alex Conconi and Claudio Gentile
SIAM Journal on Computing Volume 34, Number 3, pp. 640-668, 2005.

Abstract

Kernel-based linear-threshold algorithms, such as Support Vector Machines and Perceptron-like algorithms, are among the best available techniques for solving pattern classification problems. In this paper, we describe an extension of the classical Perceptron algorithm, called second-order Perceptron, and analyze its performance within the mistake bound model of on-line learning. The bound achieved by our algorithm depends on the sensitivity to second-order data information, and is the best known mistake bound for (efficient) kernel-based linear-threshold classifiers to date. This mistake bound, which strictly generalizes the well-known Perceptron bound, is expressed in terms of the eigenvalues of the empirical data correlation matrix and depends on a parameter controlling the sensitivity of the algorithm to the distribution of these eigenvalues. Since the optimal setting of this parameter is not known a priori, we also analyze two variants of the second-order Perceptron algorithm: one that adaptively sets the value of the parameter in terms of the number of mistakes made so far, and one parameterless, based on pseudoinverses.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:1238
Deposited By:Nicolò Cesa-Bianchi
Deposited On:28 November 2005