PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Suboptimal behaviour of Bayes and MDL in classification under misspecification
Peter Grünwald and John Langford
under submission 2005.

There is a more recent version of this eprint available. Click here to view it.

Abstract

We show that forms of Bayesian and MDL inference that are often applied to classification problems can be {\em inconsistent}. This means that there exists a learning problem such that for all amounts of data the generalization errors of the MDL classifier and the Bayes classifier relative to the Bayesian posterior both remain bounded away from the smallest achievable generalization error. We extensively discuss the result from both a Bayesian and an MDL perspective.

EPrint Type:Article
Additional Information:Under submission to a journal. Substantially extended versions of 2004 COLT conference publication.
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
ID Code:1323
Deposited By:Peter Grünwald
Deposited On:28 November 2005

Available Versions of this Item