Suboptimal behaviour of Bayes and MDL in classification under misspecification
Peter Grünwald and John Langford
In: Seventeenth Annual Conference on Computational Learning Theory (COLT 2004), 8-11 July 2004, Banff, Canada.

Abstract

We show that the MDL Principle and the Bayesian posterior in a form often applied to classification problems can be {\em inconsistent}. This means there exists a learning problem such that for all amounts of data the generalization error of the classifier selected by MDL or the Bayesian posterior remains bounded away from the smallest achievable generalization error.

Postscript - Requires a viewer, such as GhostView

[Edit]