Taylor-based pseudo-metrics for random process fitting in dynamic programming : expected loss minimisation and risk management
Sylvain Gelly, Jeremie Mary and Olivier Teytaud
In: PDMIA 2005, Lille, France(2005).
Stochastic optimization is the research of x optimizing E C (x, A), the expec-
tation of C (x, A), where A is a random variable. Typically C (x, a) is the cost
related to a strategy x which faces the realization a of the random process.
Many stochastic optimization problems deal with multiple time steps, lead-
ing to computationally difficult problems ; efficient solutions exist, for example
through Bellman’s optimality principle, but only provided that the random process
is represented by a well structured process, typically an inhomogeneous Marko-
vian process (hopefully with a finite number of states) or a scenario tree. The
problem is that in the general case, A is far from being Markovian. So, we look
for A , "looking like A", but belonging to a given family A which does not at all
contain A. The problem is the numerical evaluation of "A looks like A".
A classical method is the use of the Kantorovitch-Rubinstein distance or other
transportation metrics [Pﬂug, 2001], justified by straightforward bounds on the de-
viation |E C (x, A) − E C (x, A )| through the use of the Kantorovitch-Rubinstein
distance and uniform lipschitz conditions. These approaches might be better
than the use of high-level statistics [Keefer, 1994]. We propose other (pseudo-
)distances, based upon refined inequalities, guaranteeing a good choice of A .
Moreover, as in many cases, we indeed prefer the optimization with risk manage-
ment, e.g. optimization of E C (x, noise(A)) where noise(.) is a random noise
modelizing the lack of knowledge on the precise random variables, we propose
distances which can deal with a user-defined noise. Tests on artificial data sets
with realistic loss functions show the relevance of the method.