PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Mutual information based dimensionality reduction with application to non-linear regression
Lev Faivishevsky and Jacob Goldberger
In: MLSP 2010, 29 Aug - 1 Sep 2010, Finland.


In this paper we introduce a supervised linear dimensionality reduction algorithm which is based on finding a projected input space that maximizes mutual information between input and output values. The algorithm utilizes the recently introduced MeanNN estimator for differential entropy. We show that the estimator is an appropriate tool for the dimensionality reduction task. Next we provide a nonlinear regression algorithm based on the proposed dimensionality reduction approach. The regression algorithm achieves comparable to state-of-the-art performance on the standard datasets being three orders of magnitude faster. In addition we demonstrate an application of the proposed dimensionality reduction algorithm to reduced-complexity classification.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
ID Code:7748
Deposited By:Jacob Goldberger
Deposited On:17 March 2011