PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A general definition of conditional information and its application to ergodic decomposition
Lukasz Debowski
Statistics and Probability Letters 2008. ISSN 0167-7152


We discuss a simple definition of conditional mutual information (CMI) for fields and $\sigma$-fields. The new definition is applicable also in nonregular cases, unlike the well-known but more restricted definition of CMI by Dobrushin. Certain properties of the two notions of CMI and their equivalence for countably generated $\sigma$-fields are established. We also consider an application, which concerns the ergodic decomposition of mutual information for stationary processes. In this case, CMI is tightly linked, via additivity of information, with entropy defined as self-information. Thus we reconsider the latter concept in some detail.

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
ID Code:4833
Deposited By:Peter Grünwald
Deposited On:24 March 2009