PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Model Selection Under Covariate Shift
Masashi Sugiyama and Klaus-Robert Müller
(2005) LNCS , Volume 3697 . Springer , 15th International Conference, Warsaw, Poland, September 11-15 .

Abstract

A common assumption in supervised learning is that the training and test input points follow the same probability distribution. However, this assumption is not fulfilled, e.g., in interpolation, extrapolation, or active learning scenarios. The violation of this assumption---known as the covariate shift---causes a heavy bias in standard generalization error estimation schemes such as cross-validation and thus they result in poor model selection. In this paper, we therefore propose an alternative estimator of the generalization error. Under covariate shift, the proposed generalization error estimator is unbiased if the learning target function is included in the model at hand and it is asymptotically unbiased in general. Experimental results show that model selection with the proposed generalization error estimator is compared favorably to cross-validation in extrapolation.

EPrint Type:Book
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:1889
Deposited By:Klaus-Robert Müller
Deposited On:29 December 2005