Sparse Gaussian processes using pseudo-inputs
Ed Snelson and Zoubin Ghahramani
In: NIPS 2005, 5-10 Dec 2005, Vancouver, Canada.
We present a new Gaussian process (GP) regression model whose
covariance is parameterized by the the locations of M pseudo-input
points, which we learn by a gradient based optimization. We take M<N, where N is the number of real data points, and hence obtain a
sparse regression method which has O(M^2 N) training cost
and O(M^2) prediction cost per test case. We also find
hyperparameters of the covariance function in the same joint
optimization. The method can be viewed as a Bayesian regression model
with particular input dependent noise. The method turns out to be closely related to several other sparse GP
approaches, and we discuss the relation in detail. We finally
demonstrate its performance on some large data sets, and make a direct
comparison to other sparse GP methods. We show that our method can
match full GP performance with small M, i.e. very sparse solutions,
and it significantly outperforms other approaches in this regime.