Mind the duality gap: Logarithmic regret algorithms for online optimization
Sham Kakade and Shai Shalev-Shwartz
In: NIPS 2008, Dec 2008, Vancouver.

Abstract

We describe a primal-dual framework for the design and analysis of online strongly convex optimization algorithms. Our framework yields the tightest known logarithmic regret bounds for Follow-The-Leader and for the gradient descent algorithm proposed in \cite{HazanKaKaAg06}. We then show that one can interpolate between these two extreme cases. In particular, we derive a new algorithm that shares the computational simplicity of gradient descent but achieves lower regret in many practical situations. Finally, we further extend our framework for generalized strongly convex functions.

 PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type: Conference or Workshop Item (Paper) Project Keyword UNSPECIFIED Theory & Algorithms 5422 Shai Shalev-Shwartz 02 July 2009