PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Using Mathematical Programming to Solve Factored Markov Decision Processes with Imprecise Probabilities
K Delgado Valdivia, L de Barros, F Cozman and Scott Sanner
International Journal of Approximate Reasoning Volume 1, Number 1, pp. 1-30, 2011.


This paper investigates Factored Markov Decision Processes with Imprecise Probabilities (MDPIPs); that is, Factored Markov Decision Processes (MDPs) where transition probabilities are imprecisely specified. We derive efficient approximate solutions for Factored MDPIPs based on mathematical programming. To do this, we extend previous linear programming approaches for linear approximations in Factored MDPs, resulting in a multilinear formulation for robust “maximin” linear approximations in Factored MDPIPs. By exploiting the factored structure in MDPIPs we are able to demonstrate orders of magnitude reduction in solution time over standard exact non-factored approaches in exchange for relatively low approximation errors on a difficult class of benchmark problems with millions of states.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
ID Code:8998
Deposited By:Wray Buntine
Deposited On:21 February 2012