Superior Multi-class Classification through Margin-Optimized Single Binary Problem
Ran El-Yaniv, Dmitry Pechyony and Elad Yom-Tov
Journal of Machine Learning Research 2006.

Abstract

The problem of multiclass-to-binary reductions in the context of classification with kernel machines continues to attract considerable attention. Indeed, the current understanding of this problem is rather limited. Despite the multitude of proposed solutions no single method is known to be consistently superior to others. We developed a new multi-class classification method that reduces the multi-class problem to a single binary classifier (SBC). Our method constructs the binary problem by embedding smaller binary problems into a single space. We show that the construction of a good embedding, which allows for large margin classification, can be reduced to the task of learning linear combinations of kernels. We observe that a known margin generalization error-bound for standard binary classification applies to our construction. Our empirical examination of the new method indicates that it can outperform one-vs-all, all-pairs and the error-correcting output coding scheme.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]