PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Fast Bayesian Support Vector Machine parameter tuning with the Nystrom method
Carl Gold and Peter Sollich
Proceedings of the International Joint Conference on Neural Networks 2005 2005.


We experiment with speeding up a Bayesian method for tuning the hyperparameters of a Support Vector Machine (SVM) classifier. The Bayesian approach gives the gradients of the evidence as averages over the posterior, which can be approximated using Hybrid Monte Carlo simulation (HMC). By using the Nystrom approximation to the SVM kernel, our method significantly reduces the dimensionality of the space to be simulated in the HMC. We show that this speeds up the running time of the HMC simulation from O(n^2) (with a large prefactor) to effectively O(n), where n is the number of training samples. We conclude that the Nystrom approximation has an almost insignificant effect on the performance of the algorithm when compared to the full Bayesian method, and gives excellent performance in comparison with other approaches to hyperparameter tuning.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:979
Deposited By:Peter Sollich
Deposited On:26 May 2005