PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

ICA based on a Smooth Estimation of the Differential Entropy
Lev Faivishevsky and Jacob Goldberger
NIPS 2008.


In this paper we introduce the MeanNN approach for estimation of main information theoretic measures such as differential entropy, mutual information and divergence. As opposed to other nonparametric approaches the MeanNN results in smooth differentiable functions of the data samples with clear geometrical interpretation. Then we apply the proposed estimators to the ICA problem and obtain a smooth expression for the mutual information that can be analytically optimized by gradient descent methods. The improved performance of the proposed ICA algorithm is demonstrated on several test examples in comparison with state-of-the-art techniques.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:4567
Deposited By:Jacob Goldberger
Deposited On:13 March 2009