Evaluating the performance in automatic image annotation: example case by adaptive fusion of global image features
Ville Viitaniemi and Jorma Laaksonen
Signal Processing: Image Communications Volume 22, Number 6, pp. 557-568, 2007.

Abstract

In this work we consider two traditional metrics for evaluating performance in automatic image annotation, the normalised score (NS) and the precision/recall (PR) statistics, particularly in connection with a de facto standard 5000 Corel image benchmark annotation task. We also motivate and describe another performance measure, de-symmetrised termwise mutual information (DTMI), as a principled compromise between the two traditional extremes. In addition to discussing the measures theoretically, we correlate them experimentally for a family of annotation system configurations derived from the PicSOM image content analysis framework. Looking at the obtained performance figures, we notice that such kind of a system, based on adaptive fusion of numerous global image features, clearly outperforms the considered methods in literature.

[Edit]