|
Mind the duality gap: Logarithmic regret algorithms for online optimization
Sham Kakade and Shai Shalev-Shwartz
In: NIPS 2008, Dec 2008, Vancouver.
Abstract We describe a primal-dual framework for the design and analysis of
online strongly convex optimization algorithms. Our framework
yields the tightest known logarithmic regret bounds for
Follow-The-Leader and for the gradient descent algorithm proposed in
\cite{HazanKaKaAg06}. We then show that one can interpolate between
these two extreme cases. In particular, we derive a new algorithm
that shares the computational simplicity of gradient descent but
achieves lower regret in many practical situations. Finally, we
further extend our framework for generalized strongly convex
functions.
[Edit]
| 
