A Probabilistic Interpretation of Precision, Recall and F-score, with Implication for Evaluation
Cyril Goutte and Eric Gaussier
In: 27th European Conference on Information Retrieval (ECIR'05), March 21-23, Santiago de Compostela, Spain.
We address the problems of 1/ assessing the confidence of the
standard point estimates, precision, recall and F-score, and 2/
comparing the results, in terms of precision, recall and F-score,
obtained using two different methods. To do so, we use a probabilistic setting which allows us to obtain posterior distributions on these performance indicators, rather than point estimates. This
framework is applied to the case where different methods are run on
different datasets from the same source, as well as the standard
situation where competing results are obtained on the same data.