PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

From Empirical Bayes to Leaving-One-Out
Jesús Andrés-Ferrer and H. Ney
In: IWSM 2011(2011).


Empirical Bayes (EB) is a very appealing technique for tasks in which many outcomes of the population do not occur in the sample. In these tasks, it is necessary to estimate sparse probabilities and the EB is known to provide good estimates. However, EB estimates have two main drawbacks: they may be non-monotonic and under some circumstances they cannot be computed. This work presents a framework to constrain EB method by means of its equiva- lence with the leaving-one-out estimation. Two solutions are derived that amend the previous problems by applying two different sets of constraints: interval and monotonic. The typical application (for us) is language modelling.

EPrint Type:Conference or Workshop Item (Poster)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Learning/Statistics & Optimisation
Natural Language Processing
ID Code:8795
Deposited By:Alfons Juan
Deposited On:21 February 2012