PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Efficient Discriminative Learning of Parametric Nearest Neighbor Classifiers
Ziming Zhang, Paul Sturgess, Sunando Sengupta, Nigel Crook and Philip Torr
In: CVPR 2012, 18-20 June 2012., Rhode Island.

There is a more recent version of this eprint available. Click here to view it.


Linear SVMs are efficient in both training and testing, however the data in real applications is rarely linearly separable. Non-linear kernel SVMs are too computationally intensive for applications with large-scale data sets. Recently locally linear classifiers have gained popularity due to their efficiency whilst remaining competitive with kernel methods. The vanilla nearest neighbor algorithm is one of the simplest locally linear classifiers, but it lacks robustness due to the noise often present in real-world data. In this paper, we introduce a novel local classifier, Parametric Nearest Neighbor (P-NN) and its extension Ensemble of P-NN (EP-NN). We parameterize the nearest neighbor algorithm based on the minimum weighted squared Euclidean distances between the data points and the prototypes, where a prototype is represented by a locally linear combination of some data points. Meanwhile, our method attempts to jointly learn both the prototypes and the classifier parameters discriminatively via max-margin. This makes our classifiers suitable to approximate the classification decision boundaries locally based on nonlinear functions. During testing, the computational complexity of both classifiers is linear in the product of the dimension of data and the number of prototypes. Our classification results on MNIST, USPS, LETTER, and Chars74K are comparable and in some cases are better than many other methods such as the state-of-the-art locally linear classifiers.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
ID Code:9545
Deposited By:Sunando Sengupta
Deposited On:22 May 2012

Available Versions of this Item