PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Comparison of Independent Component Analysis and Singular Value Decomposition in Word Context Analysis
Jaakko Väyrynen and Timo Honkela
In: AKRR'05, International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reasoning, 15-17 Jun 2005, Espoo, Finland.


In earlier studies we have been able show that independent component analysis is able to extract automatically meaningful linguistic features. The emergent syntactic and semantic features are based on an analysis of the words in their contexts in large corpora. We have also shown that there is a reasonably strong correlation between traditional features and categories defined by linguists and the emergent features. In this article, we introduce a new measure for comparing the emergent and the traditionally defined features. We apply this measure to compare the emergent features produced by singular value decomposition (SVD) and independent component analysis (ICA). The conclusion is that the ICA-based features correspond to the human intuitions much more closely than the SVD-based features not only in a visual inspection but also in a systematic and principled comparison.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Natural Language Processing
ID Code:1814
Deposited By:Timo Honkela
Deposited On:28 November 2005