On the Complexity of Working Set Selection
Hans Simon
In: ALT 2004, 02-05 Oct 2004, Padova, Italy.

Abstract

The decomposition method is currently one of the major methods for solving the convex quadratic optimization problems being associated with Support Vector Machines (SVM-optimization). A subtle issue in these methods is the policy for working set selection. We would like to find policies that realize (as good as possible) three goals simultaneously: ``(fast) convergence to an optimal solution'', ``efficient procedures for working set selection'', and ``high degree of generality'' (including typical variants of SVM-optimization as special cases). Recently, a variant of the decomposition method has been presented that basically converges for any convex quadratic optimization problem provided that an appropriate ``general policy'' for working set selection is applied. In this paper, we study the computational complexity of this general policy when it is used for SVM-optimzation. We show that it poses an NP-complete working set selection problem, but a slight variation of it (sharing the convergence properties with the original policy) can be effciently solved. We show furthermore that so-called ``rate certifying pairs'' (introduced by Hush and Scovel) can be found in linear time, which leads to a quite efficient decomposition method with a polynomial convergence rate for SVM-optimization.

Postscript - Requires a viewer, such as GhostView

[Edit]