PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Universal similarity
P.M.B. Vitanyi
roc. ITW2005 - IEEE ITSOC Information Theory Workshop 2005 on Coding and Complexity, 29th Aug. - 1st Sept., 2005, Rotorua, New Zealand. 2005.


We develop rate-distortion theory for individual data with respect to general distortion measures, that is, a theory of lossy compression of individual data. This is applied to Euclidean distortion, Hamming distortion, Kolmogorov distortion, and Shannon-Fano distortion. We show that in all these cases for every function satisfying the obvious constraints there are data that have this function as their individual rate-distortion function. Shannon's distortion-rate function over a random source is shown to be the pointswise asymptotic expectation of the individual distortion-rate functions we have defined. The great differences in the distortion-rate functions for individual non-random (that is, the aspects important to lossy compression) data we established were previously invisible and obliterated in the Shannon theory. The techniques are based on Kolmogorov complexity.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Natural Language Processing
Information Retrieval & Textual Information Access
ID Code:1910
Deposited By:Paul Vitányi
Deposited On:29 December 2005