PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

The Variational Gaussian Approximation Revisited.
Manfred Opper and Cedric Archambeau
Neural Computation Volume 21, pp. 786-792, 2009.


The variational approximation of posterior distributions by multivariate Gaussians has been much less popular in the Machine Learning community compared to the corresponding approximation by factorising distributions. This is for a good reason: the Gaussian approximation is in general plagued by an $\Ocal(N^2)$ number of variational parameters to be optimised, $N$ being the number of random variables. In this work, we discuss the relationship between the Laplace and the variational approximation and we show that for models with Gaussian priors and factorising likelihoods, the number of variational parameters is actually $\Ocal(N)$. The approach is applied to Gaussian process regression with non-Gaussian likelihoods.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
ID Code:5264
Deposited By:Manfred Opper
Deposited On:24 March 2009