PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Near-optimal rates for limited-delay universal lossy source coding
Andras Gyorgy and Gergely Neu
In: 2011 IEEE International Symposium on Information Theory (ISIT), 31 July - 5 Aug. 2011, St. Petersburg, Russia.

There is a more recent version of this eprint available. Click here to view it.


We consider the problem of limited-delay lossy coding of individual sequences. Here the goal is to design (fixed-rate) compression schemes to minimize the normalized expected distortion redundancy relative to a reference class of coding schemes, measured as the difference between the average distortion of the algorithm and that of the best coding scheme in the reference class. In compressing a sequence of length T, the best schemes available in the literature achieve an O(T^{-1/3}) normalized distortion redundancy relative to finite reference classes of limited delay and limited memory. It has also been shown that the distortion redundancy is at least of order 1/√T in certain cases. In this paper we narrow the gap between the upper and lower bounds, and give a compression scheme whose distortion redundancy is O(√(ln(T)/T) ), only a logarithmic factor larger than the lower bound. The method is based on the recently introduced Shrinking Dartboard prediction algorithm, a variant of the exponentially weighted average prediction. Our method is also applied to the problem of zero-delay scalar quantization, where O(ln(T)/√T) distortion redundancy is achieved relative to the (infinite) class of scalar quantizers of a given rate, almost achieving the known lower bound of order 1/√T.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:8470
Deposited By:Andras Gyorgy
Deposited On:23 January 2012

Available Versions of this Item