PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Mind the duality gap: Logarithmic regret algorithms for online optimization
Sham Kakade and Shai Shalev-Shwartz
In: NIPS 2008, Dec 2008, Vancouver.


We describe a primal-dual framework for the design and analysis of online strongly convex optimization algorithms. Our framework yields the tightest known logarithmic regret bounds for Follow-The-Leader and for the gradient descent algorithm proposed in \cite{HazanKaKaAg06}. We then show that one can interpolate between these two extreme cases. In particular, we derive a new algorithm that shares the computational simplicity of gradient descent but achieves lower regret in many practical situations. Finally, we further extend our framework for generalized strongly convex functions.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:5422
Deposited By:Shai Shalev-Shwartz
Deposited On:02 July 2009