PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Hierarchical Multi-Stream Posterior Based Speech Recognition System
Hamed Ketabdar, Jithendra Vepa, Samy Bengio and Hervé Bourlard
In: Machine Learning for Multimodal Interactions, Second International Workshop, MLMI, 2005, Edinburgh.


In this paper, we present initial results towards boosting posterior based speech recognition systems by estimating more informative posteriors using multiple streams of features and taking into account acoustic context (e.g., as available in the whole utterance), as well as possible prior information (such as topological constraints). These posteriors are estimated based on ``state gamma posterior'' definition (typically used in standard HMMs training) extended to the case of multi-stream HMMs. This approach provides a new, principled, theoretical framework for hierarchical estimation/use of posteriors, multi-stream feature combination, and integrating appropriate context and prior knowledge in posterior estimates. In the present work, we used the resulting gamma posteriors as features for a standard HMM/GMM layer. On the OGI Digits database and on a reduced vocabulary version (1000 words) of the DARPA Conversational Telephone Speech-to-text (CTS) task, this resulted in significant performance improvement, compared to the state-of-the-art Tandem systems.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Talk)
Project Keyword:Project Keyword UNSPECIFIED
ID Code:1106
Deposited By:Samy Bengio
Deposited On:29 September 2005