Neighbourhood components analysis
Jacob Goldberger, Sam Roweis, Geoff Hinton and Ruslan Salakhutdinov
In: NIPS 2004, 13-18 Dec 2004, Vancover, Canada.
In this paper we propose a novel method for learning a
Mahalanobis distance measure to be used in the KNN classification
algorithm. The algorithm directly maximizes a stochastic variant of
the leave-one-out KNN score on the training set. It can also
learn a low-dimensional linear embedding of labeled data that can
be used for data visualization and fast classification.
Unlike other methods, our classification model is non-parametric,
making no assumptions about the shape of the class distributions or
the boundaries between them. The performance of the method
is demonstrated on several data sets, both for metric learning and
linear dimensionality reduction.