PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Kernel methods for missing variables
Alex Smola, S V N Vishwanathan and Thomas Hofmann
In: AISTATS 2005, January 2005, Barbados.


We present methods for dealing with missing variables in the context of Gaussian Processes and Support Vector Machines. This solves an important problem which has largely been ig- nored by kernel methods: How to systemati- cally deal with incomplete data? Our method can also be applied to problems with partially observed labels as well as to the transductive setting where we view the labels as missing data. Our approach relies on casting kernel meth- ods as an estimation problem in exponen- tial families. Hence, estimation with miss- ing variables becomes a problem of comput- ing marginal distributions, and finding effi- cient optimization methods. To that extent we propose an optimization scheme which ex- tends the Concave Convex Procedure (CCP) of Yuille and Rangara jan, and present a simplified and intuitive proof of its conver- gence. We show how our algorithm can be specialized to various cases in order to effi- ciently solve the optimization problems that arise. Encouraging preliminary experimen- tal results on the USPS dataset are also pre- sented.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2037
Deposited By:Alex Smola
Deposited On:16 January 2006