PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

State-space models - from the EM algorithm to a gradient approach
R. K. Olsson, K. B. Petersen and T. Lehn-Schiøler
Neural Computation 2006.


Slow convergence is observed in the EM algorithm for linear state-space models. We propose to circumvent the problem by applying any off-the-shelf quasi-Newton-type optimizer, which operates on the gradient of the log-likelihood function. Such an algorithm is a practical alternative due to the fact that the exact gradient of the log-likelihood function can be computed by recycling components of the EM algorithm. We demonstrate the efficiency of the proposed method in three relevant instances of the linear state-space model. In high signal to noise ratios, where EM is particularly prone to converge slowly, it is shown that gradient-based learning results in a sizable reduction of the computation time.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:2690
Deposited By:Rasmus Olsson
Deposited On:22 November 2006