PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Algorithmic Information Theory
Peter Grünwald and Paul Vitányi
In: Handbook of the Philosophy of Science, Volume 8: Philosophy of Information (2008) Elsevier Science .


We introduce *algorithmic information theory*, also known as the theory of *Kolmogorov complexity*. We explain the main concepts of this quantitative approach to defining `information'. We discuss the extent to which Kolmogorov's and Shannon's information theory have a common purpose, and where they are fundamentally different. We indicate how recent developments within the theory allow one to formally distinguish between `structural' (meaningful) and `random' information as measured by the *Kolmogorov structure function*, which leads to a mathematical formalization of Occam's razor in inductive inference. We end by discussing some of the philosophical implications of the theory.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Book Section
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:4592
Deposited By:Peter Grünwald
Deposited On:13 March 2009