PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Periodic finite state controllers for efficient POMDP and DEC-POMDP planning
Joni Pajarinen and Jaakko Peltonen
In: NIPS 2011, The Twenty-Fifth Annual Conference on Neural Information Processing Systems, 12-15 Dec 2011, Granada, Spain.


Applications such as robot control and wireless communication require planning under uncertainty. Partially observable Markov decision processes (POMDPs) plan policies for single agents under uncertainty and their decentralized versions (DEC-POMDPs) find a policy for multiple agents. The policy in infinite-horizon POMDP and DEC-POMDP problems has been represented as finite state controllers (FSCs). We introduce a novel class of periodic FSCs, composed of layers connected only to the previous and next layer. Our periodic FSC method finds a deterministic finite-horizon policy and converts it to an initial periodic infinite-horizon policy. This policy is optimized by a new infinite-horizon algorithm to yield deterministic periodic policies, and by a new expectation maximization algorithm to yield stochastic periodic policies. Our method yields better results than earlier planning methods and can compute larger solutions than with regular FSCs.

EPrint Type:Conference or Workshop Item (Paper)
Additional Information:
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Computational, Information-Theoretic Learning with Statistics
Learning/Statistics & Optimisation
Theory & Algorithms
ID Code:9070
Deposited By:Jaakko Peltonen
Deposited On:21 February 2012