PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Mapping from Speech to Images Using Continuous State Space Models
Tue Lehn-Schiøler, Lars Kai Hansen and Jan Larsen
In: MLMI'04, 21-23 June 2004, Martigny, Switzerland.


In this paper a system that transforms speech waveforms to animated faces are proposed. The system relies on continuous state space models to perform the mapping, this makes it possible to ensure video with no sudden jumps and allows continuous control of the parameters in 'face space'. The performance of the system is critically dependent on the number of hidden variables, with too few variables the model cannot represent data, and with too many overfitting is noticed. Simulations are performed on recordings of 3-5 sec. video sequences with sentences from the Timit database. From a subjective point of view the model is able to construct an image sequence from an unknown noisy speech sequence even though the number of training examples are limited.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Multimodal Integration
Theory & Algorithms
ID Code:162
Deposited By:Tue Lehn-Schiøler
Deposited On:23 November 2004