PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Convergent Expectation Propagation in Linear Models with Spike-and-slab Priors
José Miguel Hernánez Lobato and Daniel Hernández-Lobato
Arxiv preprint arXiv:1112.2289 2011.


Exact inference in the linear regression model with spike and slab priors is often intractable. Expectation propagation (EP) can be used for approximate inference. However, the regular sequential form of EP (R-EP) may fail to converge in this model when the size of the training set is very small. As an alternative, we propose a provably convergent EP algorithm (PC-EP). PC-EP is proved to minimize an energy function which, under some constraints, is bounded from below and whose stationary points coincide with the solution of R-EP. Experiments with synthetic data indicate that when R-EP does not converge, the approximation generated by PC-EP is often better. By contrast, when R-EP converges, both methods perform similarly.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
ID Code:8454
Deposited By:José Miguel Hernánez Lobato
Deposited On:19 January 2012