Learned Graphical Models for Probabilistic Planning Provide a New Class of Movement Primitives
Elmar Rückert, Gerhard Neumann, Marc Toussaint and Wolfgang Maass
Frontiers in Computational Neuroscience
Biological movement generation combines three interesting aspects: its
modular organization in movement primitives, its characteristics of
stochastic optimality under perturbations, and its efficiency in terms
of learning. A common approach to motor skill learning is to
endow the primitives with dynamical systems. Here, the parameters of
the primitive indirectly define the shape of a reference
trajectory. We propose an alternative movement primitive representation based on
probabilistic inference in learned graphical models with new and interesting properties
that complies with salient features of biological movement control.
Instead of endowing the primitives with dynamical systems,
we propose to endow movement primitives with an intrinsic
probabilistic planning system, integrating the power of stochastic
optimal control methods within a movement primitive.
The parametrization of the primitive is a graphical model that represents
the dynamics and intrinsic cost function such that inference in this
graphical model yields the control policy. We parametrize the
intrinsic cost function using task-relevant features, such as the
importance of passing through certain via-points.
The system dynamics as well as intrinsic cost function parameters are learned in a
reinforcement learning setting. We evaluate our approach on a complex
4-link balancing task. Our experiments show that our movement
representation facilitates learning significantly and leads to better
generalization to new task settings without re-learning.