ICA based on a Smooth Estimation of the Differential Entropy
Lev Faivishevsky and Jacob Goldberger
In this paper we introduce the MeanNN approach for estimation
of main information theoretic measures such as differential entropy, mutual
information and divergence. As opposed to other nonparametric approaches the
MeanNN results in smooth differentiable functions of the data samples with
clear geometrical interpretation. Then we apply the proposed estimators
to the ICA problem and obtain a smooth expression for the mutual
information that can be analytically optimized by gradient descent methods.
The improved performance of the proposed ICA algorithm is demonstrated on several
test examples in comparison with state-of-the-art techniques.