PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

The Information Bottleneck Revisited or How to Choose a Good Distortion Measure
Peter Harremoes and Naftali Tishby
2007 IEEE International Symposium on Information Theory (ISIT2007) 2007.


It is well-known that the information bottleneck method and rate distortion theory are related. Here it is described how the information bottleneck can be considered as rate distortion theory for a family of probability measures where information divergence is used as distortion measure. It is shown that the information bottleneck method has some properties that are not shared with rate distortion theory based on any other divergence measure. In this sense the information bottleneck method is unique.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Theory & Algorithms
ID Code:4087
Deposited By:Naftali Tishby
Deposited On:25 February 2008