PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Independent Variable Group Analysis in Learning Compact Representations for Data
Krista Lagus, Esa Alhoniemi, Jeremias Seppä, Antti Honkela and Paul Wagner
In: AKRR'05, International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reasoning, 15-17 Jun 2005, Espoo, Finland.

Abstract

Humans tend to group together related properties in order to understand complex phenomena. When modeling large problems with limited representational resources, it is important to be able to construct compact models of the data. Structuring the problem into sub-problems that can be modeled independently is a means for achieving compactness. We describe the Independent Variable Group Analysis (IVGA), an unsupervised learning principle that in modeling a data set, also discovers a grouping of the input variables that reflects statistical independencies in the data. In addition, we discuss its connection to some aspects of cognitive modeling and of representations in the brain. The IVGA approach and its implementation are designed to be practical, efficient, and useful for real world applications. Initial experiments on several data sets are reported to examine the performance and potential uses of the method. The preliminary results are promising: the method does seem to find independent subsets of variables. Moreover, it leads to markedly more compact and efficient models than the full model without variable grouping. This allows the re-allocation of freed representational resources for other important tasks. Compact models also contain much fewer parameters and generalize better, and therefore require less data for estimation.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Talk)
Additional Information:pp. 49-56
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Theory & Algorithms
ID Code:1677
Deposited By:Krista Lagus
Deposited On:28 November 2005