PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Improvements to the Sequence Memoizer
Jan Gasthaus and Yee Whye Teh
In: Advances in Neural Information Processing Systems (2010) Neural Information Processing Systems Foundation , pp. 685-693.


The sequence memoizer is a model for sequence data with state-of-the-art performance on language modeling and compression. We propose a number of improvements to the model and inference algorithm, including an enlarged range of hyperparameters, a memory-efficient representation, and inference algorithms operating on the new representation. Our derivations are based on precise definitions of the various processes that will also allow us to provide an elementary proof of the “mysterious” coagulation and fragmentation properties used in the original paper on the sequence memoizer by Wood et al. (2009). We present some experimental results supporting our improvements.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Book Section
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Natural Language Processing
Theory & Algorithms
Information Retrieval & Textual Information Access
ID Code:7145
Deposited By:Jan Gasthaus
Deposited On:06 March 2011