PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A Theory of Lossy Compression for Individual Data
Nikolai K. Vereshchagin and Paul M.B. Vitanyi 2004.


We develop rate-distortion theory for individual data with respect to general distortion measures, that is, a theory of lossy compression of individual data. This is applied to Euclidean distortion, Hamming distortion, Kolmogorov distortion, and Shannon-Fano distortion. We show that in all these cases for every function satisfying the obvious constraints there are data that have this function as their individual rate-distortion function. Shannon's distortion-rate function over a random source is shown to be the pointswise asymptotic expectation of the individual distortion-rate functions we have defined. The great differences in the distortion-rate functions for individual non-random (that is, the aspects important to lossy compression) data we established were previously invisible and obliterated in the Shannon theory. The techniques are based on Kolmogorov complexity.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Information Retrieval & Textual Information Access
ID Code:327
Deposited By:Paul Vitányi
Deposited On:11 December 2004