roc. ITW2005 - IEEE ITSOC Information Theory Workshop 2005 on Coding and Complexity, 29th Aug. - 1st Sept., 2005, Rotorua, New Zealand.
We develop rate-distortion theory for individual data
with respect to general distortion measures, that is, a theory
of lossy compression of individual data.
This is applied to Euclidean distortion, Hamming distortion,
Kolmogorov distortion, and Shannon-Fano distortion.
We show that in all these cases for every
function satisfying the obvious constraints there
are data that have this function as their
individual rate-distortion function.
Shannon's distortion-rate function over a random source
is shown to be the pointswise asymptotic
expectation of the individual distortion-rate
functions we have defined.
The great differences
in the distortion-rate functions for individual non-random (that is,
the aspects important to lossy compression)
data we established were previously invisible
and obliterated in the Shannon theory.
The techniques are based on Kolmogorov complexity.