PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

On the Complexity of Working Set Selection
Hans Simon
In: ALT 2004, 02-05 Oct 2004, Padova, Italy.

Abstract

The decomposition method is currently one of the major methods for solving the convex quadratic optimization problems being associated with Support Vector Machines (SVM-optimization). A subtle issue in these methods is the policy for working set selection. We would like to find policies that realize (as good as possible) three goals simultaneously: ``(fast) convergence to an optimal solution'', ``efficient procedures for working set selection'', and ``high degree of generality'' (including typical variants of SVM-optimization as special cases). Recently, a variant of the decomposition method has been presented that basically converges for any convex quadratic optimization problem provided that an appropriate ``general policy'' for working set selection is applied. In this paper, we study the computational complexity of this general policy when it is used for SVM-optimzation. We show that it poses an NP-complete working set selection problem, but a slight variation of it (sharing the convergence properties with the original policy) can be effciently solved. We show furthermore that so-called ``rate certifying pairs'' (introduced by Hush and Scovel) can be found in linear time, which leads to a quite efficient decomposition method with a polynomial convergence rate for SVM-optimization.

Postscript - Requires a viewer, such as GhostView
EPrint Type:Conference or Workshop Item (Paper)
Additional Information:A full version of this extended abstract is accepted for publication in the TCS Special Issue on ALT 2004. (The full version contains additional results/proofs.)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:125
Deposited By:Hans Simon
Deposited On:27 May 2004