PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Online Learning meets Optimization in the Dual
Shai Shalev-Shwartz and Yoram Singer
In: COLT 2006, June 22-25, 2006, Pittsburgh, Pennsylvania, USA.


We describe a novel framework for the design and analysis of online learning algorithms based on the notion of duality in constrained optimization. We cast a sub-family of universal online bounds as an optimization problem. Using the weak duality theorem we reduce the process of online learning to the task of incrementally increasing the dual objective function. The amount by which the dual increases serves as a new and natural notion of progress. We are thus able to tie the primal objective value and the number of prediction mistakes using and the increase in the dual. The end result is a general framework for designing and analyzing old and new online learning algorithms in the mistake bound model.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2144
Deposited By:Shai Shalev-Shwartz
Deposited On:11 July 2006