PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A Primal-Dual Perspective of Online Learning Algorithms
Shai Shalev-Shwartz and Yoram Singer
Machine Learning Journal Volume 69, Number 2, pp. 115-142, 2007. ISSN 1573-0565


We describe a novel framework for the design and analysis of online learning algorithms based on the notion of duality in constrained optimization. We cast a sub-family of universal online bounds as an optimization problem. Using the weak duality theorem we reduce the process of online learning to the task of incrementally increasing the dual objective function. The amount by which the dual increases serves as a new and natural notion of progress for analyzing online learning algorithms. We are thus able to tie the primal objective value and the number of prediction mistakes using the increase in the dual.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:4061
Deposited By:Shai Shalev-Shwartz
Deposited On:25 February 2008