Algorithmic Information Theory
Peter Grünwald and Paul Vitányi
Handbook of the Philosophy of Science, Volume 8: Philosophy of Information
We introduce *algorithmic information theory*, also known as the theory of *Kolmogorov complexity*. We explain the main concepts of this quantitative approach to defining `information'. We discuss the extent to which Kolmogorov's and Shannon's information theory have a common purpose, and where they are fundamentally different. We indicate how recent developments within the theory allow one to formally distinguish between `structural' (meaningful) and `random' information as measured by the *Kolmogorov structure function*, which leads to a mathematical formalization of Occam's razor in inductive inference. We end by discussing some of the philosophical implications of the theory.