PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Efficient Multi-start Strategies for Local Search Algorithms
Levente Kocsis and Andras Gyorgy
In: Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases , ECML PKDD 2009, Part I Lecture Notes in Computer Science , LNAI 5781/2009 . (2009) Springer , Berlin/Heidelberg, Germany , pp. 705-720. ISBN 978-3-642-04179-2


Local search algorithms for global optimization often suffer from getting trapped in a local optimum. The common solution for this problem is to restart the algorithm when no progress is observed. Alternatively, one can start multiple instances of a local search algorithm, and allocate computational resources (in particular, processing time) to the instances depending on their behavior. Hence, a multi-start strategy has to decide (dynamically) when to allocate additional resources to a particular instance and when to start new instances. In this paper we propose a consistent multi-start strategy that assumes a convergence rate of the local search algorithm up to an unknown constant, and in every phase gives preference to those instances that could converge to the best value for a particular range of the constant. Combined with the local search algorithm SPSA (Simultaneous Perturbation Stochastic Approximation), the strategy performs remarkably well in practice, both on synthetic tasks and on tuning the parameters of learning algorithms.

EPrint Type:Book Section
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
ID Code:6134
Deposited By:Andras Gyorgy
Deposited On:08 March 2010