PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Caveats on Bayesian Derivations of NML Probabilities: A Comment on Karabatsos and Walker (2006)
Peter Grünwald and Daniel Navarro
Mathematical Psychology Volume 53, pp. 43-51, 2009.


The normalized maximum likelihood criterion for selecting among competing models is generally justified on information-theoretic grounds, via the principle of minimum description length (MDL). In a recent paper, however, Karabatsos and Walker (2006) (KW from now on) propose an alternative Bayesian decision-theoretic characterization, in which one part of the criterion (the likelihood term) arises from placing a Dirichlet process prior over possible data-generating distributions, and the other part (the complexity term) is folded into a loss function. Whereas in the original derivations of NML, the complexity term arises naturally, in the KW derivation its mathematical form is taken for granted and not explained any further. We argue that for this reason, the KW characterization is incomplete; relatedly, we also question the relevance of the characterization and the validity of its consequences.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:3329
Deposited By:Peter Grünwald
Deposited On:07 February 2008