The Value of Observation for Monitoring Dynamic Systems
Eyal Even-Dar, Sham Kakade and Yishay Mansour
In: IJCAI 2007, Jan 6-12, 2007, hyderabad, India.

Abstract

We consider the fundamental problem of tracking the belief state in a POMDP, when the model is only approximately correct and the initial belief state might be unknown. In this general setting where the model is (perhaps only slightly) mis-specified, tracking (and consequently planning) may be impossible as errors accumulate. We provide a new characterization, the \emph{value of observation}, which allows us to bound the error accumulation when the initial state is unknown and the model is only approximate. The value of observation is a parameter that governs how much information the observation contains. When it is 1 the POMDP is an MDP while for an unobservable MDP the parameter is 0. Thus, the new parameter characterizes a spectrum from MDPs to unobservable MDPs depending on the amount of information conveyed in the observations.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.

[Edit]