PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

A New Perspective for Information Theoretic Feature Selection
Gavin Brown
In: Artificial Intelligence and Statistics, April 16-18 2009, Tampa, Florida.


Feature Filters are among the simplest and fastest approaches to feature selection. A filter defines a statistical criterion, used to rank features on how useful they are expected to be for classification. The highest ranking features are retained, and the lowest ranking can be discarded. A common approach is to use the Mutual Information between the feature and class label. This area has seen a recent flurry of activity, resulting in a confusing variety of heuristic criteria all based on mutual information, and a lack of a principled way to understand or relate them. The contribution of this paper is a unifying theoretical understanding of such filters. In contrast to current methods which manually construct filter criteria with particular properties, we show how to naturally derive space of possible ranking criteria. We will show that several recent contributions in the feature selection literature are points within this continuous space, and that there exist many points that have never been explored.

PDF (PDF) - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:4767
Deposited By:Gavin Brown
Deposited On:24 March 2009