PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Dimensionality reduction based on nonparametric mutual information
Lev Faivishevsky and Jacob Goldberger
Neurocomputing Volume 80, pp. 31-37, 2012.


In this paper we introduce a supervised linear dimensionality reduction algorithm which finds a projected input space that maximizes the mutual information between input and output values. The algorithm utilizes the recently introduced MeanNN estimator for differential entropy. We show that the estimator is an appropriate tool for the dimensionality reduction task. Next we provide a nonlinear regression algorithm based on the proposed dimensionality reduction approach. The regression algorithm achieves comparable to state-of-the-art performance on the standard datasets but is three orders of magnitude faster. In addition we describe applications of the proposed dimensionality reduction algorithm to reduced-complexity supervised and semisupervised classification tasks.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
ID Code:8803
Deposited By:Jacob Goldberger
Deposited On:21 February 2012