PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

On the Complexity of Working Set Selection
Hans Simon
Theoretical Computer Science 2006.

Abstract

The decomposition method is currently one of the major methods for solving the convex quadratic optimization problems being associated with Support Vector Machines (SVM-optimization). A key issue in this approach is the policy for working set selection. We would like to find policies that realize (as good as possible) three goals simultaneously: ``(fast) convergence to an optimal solution'', ``efficient procedures for working set selection'', and ``high degree of generality'' (including typical variants of SVM-optimization as special cases). In this paper, we study a general policy for working set selection that has been proposed recently. It is known that it efficiently approaches feasible solutions of minimum cost for any convex quadratic optimization problem. Here, we investigate its computational complexity when it is used for SVM-optimization. It turns out that, for variable size of the working set, the general policy poses an NP-hard working set selection problem. But a slight variation of it (sharing the convergence properties with the original policy) can be solved in polynomial time. For working sets of fixed size 2, the situation is even better. In this case, the general policy coincides with the ``rate certifying pair approach'' (introduced by Hush and Scovel). We show that maximum rate certifying pairs can be found in linear time, which leads to a quite efficient decomposition method with a polynomial convergence rate for SVM-optimization.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Additional Information:An extended abstract of this paper (with less complete proofs and only partial results) was published in the Proceedings of ALT 2004.
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2866
Deposited By:Hans Simon
Deposited On:22 November 2006