PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Active Learning of Model Evidence Using Bayesian Quadrature
Michael Osborne, David Duvenaud, Roman Garnett, Stephen Roberts, Carl Edward Rasmussen and Zoubin Ghahramani
In: Neural Information Processing Systems, 3-9 Dec 2012, Lake Tahoe, California, USA.

Abstract

Numerical integration is a key component of many problems in scientific computing, statistical modelling, and machine learning. Bayesian Quadrature is a model-based method for numerical integration which, relative to standard Monte Carlo methods, offers increased sample efficiency and a more robust estimate of the uncertainty in the estimated integral. We propose a novel Bayesian Quadrature approach for numerical integration when the integrand is non-negative, such as the case of computing the marginal likelihood, predictive distribution, or normalising constant of a probabilistic model. Our approach approximately marginalises the quadrature model's hyperparameters in closed form, and introduces an active learning scheme to optimally select function evaluations, as opposed to using Monte Carlo samples. We demonstrate our method on both a number of synthetic benchmarks and a real scientific problem from astronomy.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:9633
Deposited By:David Duvenaud
Deposited On:09 December 2012