PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Unsupervised Models for Morpheme Segmentation and Morphology Learning
Mathias Creutz and Krista Lagus
ACM Transactions on Speech and Language Processing 2005.


We present a model family called Morfessor for the unsupervised induction of a simple morphology from raw text data. The model is formulated in a probabilistic maximum a posteriori framework. Morfessor can handle highly-inflecting and compounding languages, where words can consist of lengthy sequences of morphemes. A lexicon of word segments, so called morphs, is induced from the data. The lexicon stores information about both the usage and form of the morphs. Several instances of the model are evaluated quantitatively in a morpheme segmentation task on different sized sets of Finnish as well as English data. Morfessor is shown to perform very well compared to a widely known benchmark algorithm, in particular on Finnish data.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Natural Language Processing
ID Code:2394
Deposited By:Mathias Creutz
Deposited On:22 November 2006