PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Mutual Information Criteria for Feature Selection
Zhihong Zhang and Edwin Hancock
In: SIMBAD - Similarity-Based Pattern Recognition - First International Workshop, September 28-30, 2011, Venice, Italy.


In many data analysis tasks, one is often confronted with very high dimensional data. The feature selection problem is essentially a combinatorial optimization problem which is computationally expensive. To overcome this problem it is frequently assumed either that features independently influence the class variable or do so only involving pairwise feature interaction. In prior work [18], we have explained the use of a new measure called multidimensional interaction information (MII) for feature selection. The advantage of MII is that it can consider third or higher order feature interaction. Using dominant set clustering, we can extract most of the informative features in the leading dominant sets in advance, limiting the search space for higher order interactions. In this paper, we provide a comparison of different similarity measures based on mutual information. Experimental results demonstrate the effectiveness of our feature selection method on a number of standard data-sets.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Machine Vision
ID Code:8562
Deposited By:Edwin Hancock
Deposited On:13 February 2012