PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Bayes and Tukey meet at the Center Point
Ran Gilad-Bachrach, Amir Navot and Naftali Tishby
Proceedings of the ACM Conference on Comptational Learning Theory Volume 17, 2004.


The Bayes classifier achieves the minimal error rate by constructing a weighted majority over all concepts in the concept class. The "Bayes Point" uses the single concept in the class which has the minimal error. This way, the Bayes Point avoids some of the deficiencies of the Bayes classifier. We prove a bound on the generalization error for Bayes Point Machines when learning linear classifiers and show that it is at most ~1.71 times the generalization error of the Bayes classifier, independent of the input dimension and length of training. We show that when learning linear classifiers, the Bayes Point is almost identical to the Tukey Median and Center Point. We extend these definitions beyond linear classifiers and define the Bayes Depth of a classifier. We prove generalziation bound in terms of this new definition. Finally, we provide a new concentration of measure inequality for multivariate random variables to the Tukey Median.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Additional Information:Paper won the COLT 2004 best student paper award.
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:2001
Deposited By:Naftali Tishby
Deposited On:14 January 2006