PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Evaluating performance of automatic image annotation: example case by fusing global image features
Ville Viitaniemi and Jorma Laaksonen
In: Fifth International Workshop on Content-Based Multimedia Indexing (CBMI 2007), 25-27 Jun 2007, Bordeaux, France.


In this paper we consider two traditional metrics for evaluating the performance in automatic image annotation, the normalised score (NS) and the precision/recall (PR) statistics, particularly in connection with a de facto standard 5000 Corel image benchmark annotation task. We also motivate and describe a third performance measure, de-symmetrised termwise mutual information (DTMI), as a principled compromise between the two traditional extremes. In addition to discussing the measures theoretically, we correlate them experimentally for a family of annotation system configurations derived from the PicSOM image content analysis framework. Looking at the obtained performance figures, we notice that such kind of a system based on the fusion of numerous global image features clearly outperforms the considered methods in the literature.

EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Machine Vision
Information Retrieval & Textual Information Access
ID Code:3290
Deposited By:Ville Viitaniemi
Deposited On:07 February 2008