PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Online and Batch Learning of Pseudo-Metrics
Shai Shalev-Shwartz, Andrew Y. Ng and Yoram Singer
In: ICML 2004, Banff, Canada(2004).

There is a more recent version of this eprint available. Click here to view it.


We describe and analyze an online algorithm for supervised learning of pseudo-metrics. The algorithm receives pairs of instances and predicts their similarity according to a pseudo-metric. The pseudo-metrics we use are quadratic forms parameterized by positive semi-definite matrices. The core of the algorithm is an update rule that is based on successive projections onto the positive semi-definite cone and onto half-space constraints imposed by the examples. We describe an efficient procedure for performing these projections, derive a worst case mistake bound on the similarity predictions, and discuss a dual version of the algorithm in which it is simple to incorporate kernel operators. The online algorithm also serves as a building block for deriving a large-margin batch algorithm. We demonstrate the merits of the proposed approach by conducting experiments on MNIST dataset and on document filtering.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:60
Deposited By:Shai Shalev-Shwartz
Deposited On:14 May 2004

Available Versions of this Item