Bayesian Estimators for Robins-Ritov's Problem
Stefan Harmeling and Marc Toussaint
University of Edinburgh, School of Informatics Research Report.
Bayesian or likelihood-based approaches to data analysis became very popular in the field
of Machine Learning. However, there exist theoretical results which question the general applicability
of such approaches; among those a result by Robins and Ritov which introduce a specific example for
which they prove that a likelihood-based estimator will fail (i.e. it does for certain cases not converge
to a true parameter estimate, even given infinite data). In this paper we consider various approaches to
formulate likelihood-based estimators in this example, basically by considering various extensions of the
presumed generative model of the data. We can derive estimators which are very similar to the classical
Horvitz-Thompson and which also account for a priori knowledge of an observation probability function.