PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Efficient Unsupervised Discovery of Word Categories Using Symmetric Patterns and High Frequency Words
Dmitry Davidov and Ari Rappoport
ACL 2006 2006.


We present a novel approach for discovering word categories, sets of words sharing a significant aspect of their meaning. We utilize meta-patterns of highfrequency words and content words in order to discover pattern candidates. Symmetric patterns are then identified using graph-based measures, and word categories are created based on graph clique sets. Our method is the first pattern-based method that requires no corpus annotation or manually provided seed patterns or words. We evaluate our algorithm on very large corpora in two languages, using both human judgments and WordNetbased evaluation. Our fully unsupervised results are superior to previous work that used a POS tagged corpus, and computation time for huge corpora are orders of magnitude faster than previously reported.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Natural Language Processing
ID Code:4095
Deposited By:Ari Rappoport
Deposited On:25 March 2008