PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

On the working set selection in gradient projection-based decomposition techniques for support vector machines
Thomas Serafini and Luca Zanni
Optimization Methods and Software Volume 20, Number 4-5, pp. 583-596, 2005. ISSN 1055-6788

Abstract

This work deals with special decomposition techniques for the large quadratic program arising in training Support Vector Machines. These approaches split the problem into a sequence of quadratic programming subproblems which can be solved by efficient gradient projection methods recently proposed. By decomposing into much larger subproblems than standard decomposition packages, these techniques show promising performance and are well suited for parallelization. Here, we discuss a crucial aspect for their effectiveness: the selection of the working set, that is the index set of the variables to be optimized at each step through the quadratic programming subproblem. We analyse the most popular working set selections and develop a new selection strategy that improves the convergence rate of the decomposition schemes based on large sized working sets. The effectiveness of the proposed strategy within the gradient projection-based decomposition techniques is shown by numerical experiments on large benchmark problems, both in serial and parallel environments.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:3402
Deposited By:Luca Zanni
Deposited On:10 February 2008