PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation
Masashi Suigyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau and Motoaki Kawanabe
In: NIPS 2007, 3-8 Dec 2007, Vancouver, Canada.


A situation where training and test samples follow different input distributions is called covariate shift. Under covariate shift, standard learning methods such as maximum likelihood estimation are no longer consistent—weighted variants ac- cording to the ratio of test and training input densities are consistent. Therefore, accurately estimating the density ratio, called the importance, is one of the key is- sues in covariate shift adaptation. A naive approach to this task is to first estimate training and test input densities separately and then estimate the importance by taking the ratio of the estimated densities. However, this naive approach tends to perform poorly since density estimation is a hard task particularly in high dimen- sional cases. In this paper, we propose a direct importance estimation method that does not involve density estimation. Our method is equipped with a natural cross validation procedure and hence tuning parameters such as the kernel width can be objectively optimized. Simulations illustrate the usefulness of our approach.

EPrint Type:Conference or Workshop Item (Poster)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:3287
Deposited By:Paul Buenau
Deposited On:16 March 2012