Efﬁcient Discriminative Learning of Parametric Nearest Neighbor Classiﬁers
There is a more recent version of this eprint available. Click here to view it.
Linear SVMs are efﬁcient in both training and testing, however the data in real applications is rarely linearly separable. Non-linear kernel SVMs are too computationally intensive for applications with large-scale data sets. Recently locally linear classiﬁers have gained popularity due to their efﬁciency whilst remaining competitive with kernel methods. The vanilla nearest neighbor algorithm is one of the simplest locally linear classiﬁers, but it lacks robustness due to the noise often present in real-world data. In this paper, we introduce a novel local classiﬁer, Parametric Nearest Neighbor (P-NN) and its extension Ensemble of P-NN (EP-NN). We parameterize the nearest neighbor algorithm based on the minimum weighted squared Euclidean distances between the data points and the prototypes, where a prototype is represented by a locally linear combination of some data points. Meanwhile, our method attempts to jointly learn both the prototypes and the classiﬁer parameters discriminatively via max-margin. This makes our classiﬁers suitable to approximate the classiﬁcation decision boundaries locally based on nonlinear functions. During testing, the computational complexity of both classiﬁers is linear in the product of the dimension of data and the number of prototypes. Our classiﬁcation results on MNIST, USPS, LETTER, and Chars74K are comparable and in some cases are better than many other methods such as the state-of-the-art locally linear classiﬁers.
Available Versions of this Item