Propagating Uncertainty in POMDP value iteration with Gaussian Processes
Eric P Tuttle and Zoubin Ghahramani
University College London, London, UK.
In this paper, we describe the general approach of trying to solve
Partially Observable Markov Decision Processes with approximate value
iteration. Methods based on this approach have shown
promise for tackling larger problems where exact methods are doomed,
but we explain how most of them suffer from the fundamental problem of ignoring information about the uncertainty of their estimates. We
then suggest a new method for value iteration which uses Gaussian
processes to form a Bayesian representation of the uncertain POMDP
value function. We evaluate this method on several standard POMDPs
and obtain promising results.