Lifted coordinate descent for learning with trace-norm regularization
Miroslav Dudik, Zaid Harchaoui and Jerome Malick
In: AISTATS 2012, 21-23 Apr 2012, Canary Islands.

Abstract

We consider the minimization of a smooth loss with trace-norm regularization, which is a natural objective in multi-class and multi- task learning. Even though the problem is convex, existing approaches rely on optimiz- ing a non-convex variational bound, which is not guaranteed to converge, or repeat- edly perform singular-value decomposition, which prevents scaling beyond moderate ma- trix sizes. We lift the non-smooth con- vex problem into an infinitely dimensional smooth problem and apply coordinate de- scent to solve it. We prove that our approach converges to the optimum, and is competitive or outperforms state of the art.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]