PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models
Sathiya Keerthi, Vikas Sindhwani and Olivier Chapelle
In: NIPS(2007).


We consider the task of tuning hyperparameters in SVM models based on minimizing a smooth performance validation function, e.g., smoothed k-fold cross-validation error, using non-linear optimization techniques. The key computation in this approach is that of the gradient of the validation function with respect to hyperparameters. We show that for large-scale problems involving a wide choice of kernel-based models and validation functions, this computation can be very efficiently done; often within just a fraction of the training time. Empirical results show that a near-optimal set of hyperparameters can be identified by our approach with very few training rounds and gradient computations.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2626
Deposited By:Olivier Chapelle
Deposited On:16 November 2006