PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Covariate Shift by Kernel Mean Matching
A. Gretton, A.J. Smola, J. Huang, M. Schmittfull, K.M. Borgwardt and B. Schölkopf
In: Dataset Shift in Machine Learning (2009) MIT Press , Cambridge, MA, USA . ISBN 978-0-262-17005-5


Given sets of observations of training and test data, we consider the problem of re-weighting the training data such that its distribution more closely matches that of the test data. We achieve this goal by matching covariate distributions between training and test sets in a high dimensional feature space (specifically, a reproducing kernel Hilbert space). This approach does not require distribution estimation. Instead, the sample weights are obtained by a simple quadratic programming procedure. We provide a uniform convergence bound on the distance between the reweighted training feature mean and the test feature mean, a transductive bound on the expected loss of an algorithm trained on the reweighted data, and a connection to single class SVMs. While our method is designed to deal with the case of simple covariate shift (in the sense of Chapter ??), we have also found benefits for sample selection bias on the labels. Our correction procedure yields its greatest and most consistent advantages when the learning algorithm returns a classifier/regressor that is \simpler" than the data might suggest.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Book Section
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Brain Computer Interfaces
Theory & Algorithms
ID Code:4354
Deposited By:Bernhard Schölkopf
Deposited On:13 March 2009