Consistency of Gaussian Process Prediction
Christopher Williams
In: Notions of Complexity: Information-theoretic, Computational and Statistical Approaches Workshop, 7-9 October 2004, Eindhoven, The Netherlands.

Abstract

The main aim of this talk is to raise the issue of the consistency of Gaussian Process (GP) predictors with the other workshop participants. GP prediction works by placing a stochastic Gaussian process prior over functions and conditioning this on observations in order to make predictions. There are close similarities between GPs, Support Vector Machines (SVMs) and other kernel machines; the kernel is identified as the covariance function of the prior process. In both the regression case (Gaussian noise) and the classification case (logistic link function) the posterior is unimodal. In the regression case the posterior is again a Gaussian process, but this is not so in the classification case. Note that for the regression case the posterior mean(=mode) is obtained by the RKHS smoothing construction with squared error. I am interested in the consistency of both regression and classification problems when the kernel is sufficiently rich (i.e. non-degenerate). One method for analyzing the consistency is through the use of the equivalent kernel (Silverman, 1984) which is obtained in an idealized situation where the datapoints are ``smeared out'' in input space at some density of observations. However, I imagine that the participants may have other powerful methods to bring to bear on this problem.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]