Kernel Methods for Missing Variables
Alex Smola, S V N Vishwanathan and Thomas Hoffman
In: AISTATS 2005, 06 - 08 Jan 2005, Barbados.
We present methods for dealing with missing variables in the context
of Gaussian Processes and Support Vector Machines. This solves an
important problem which has largely been ignored by kernel methods:
How to systematically deal with incomplete data? Our method can also
be applied to problems with partially observed labels as well as to
the transductive setting where we view the labels as missing data.
Our approach relies on casting kernel methods as an estimation
problem in exponential families. Hence, estimation with missing
variables becomes a problem of computing marginal distributions, and
finding efficient optimization methods. To that extent we propose an
optimization scheme which extends the Concave Convex Procedure (CCP)
of Yuille and Rangarajan, and present a simplified and intuitive
proof of its convergence. We show how our algorithm can be
specialized to various cases in order to efficiently solve the
optimization problems that arise. Encouraging preliminary
experimental results on the USPS dataset are also presented.