Thoughts on evaluation of image retrieval inspired by ImageCLEF 2007 object retrieval task
In this paper we discuss questions on information retrieval evaluation brought up by the analysis of the recent ImageCLEF 2007 object retrieval task. We question the adequacy of the pooling strategy used in this evaluation and discuss pooling in general. We also argue that use of MAP to rank the participants is not very meaningful in this case. Instead, the result of each query should be considered separately.