PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

LS-SVM hyperparameter selection with a nonparametric noise estimator
Amaury Lendasse, Yongnan Ji, Nima Reyhani and Michel Verleysen
In: Artificial Neural Networks: Biological Inspirations – ICANN 2005, 15th International Conference, Warsaw, Poland, September 11-15, 2005, Proceedings, Part II Lecture Notes in Computer Science , 3697 (XXXII). (2005) Springer-Verlag GmbH , Germany , pp. 625-630. ISBN 3-540-28755-8


This paper presents a new method for the selection of the two hyperparameters of Least Squares Support Vector Machine (LS-SVM) approximators with Gaussian Kernels. The two hyperparameters are the width σ of the Gaussian kernels and the regularization parameter λ. For different values of σ, a Nonparametric Noise Estimator (NNE) is introduced to estimate the variance of the noise on the outputs. The NNE allows the determination of the best λ for each given σ. A Leave-one-out methodology is then applied to select the best σ. Therefore, this method transforms the double optimization problem into a single optimization one. The method is tested on 2 problems: a toy example and the Pumadyn regression Benchmark.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Book Section
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:1666
Deposited By:Amaury Lendasse
Deposited On:28 November 2005