PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Providing Consistent and Exhaustive Relevance Assessments for XML Retrieval Evaluation
Benjamin Piwowarski and Mounia Lalmas
In: CIKM 2004, 8-13 Nov 2004, Washington DC, USA.


Comparing retrieval approaches requires test collections, which consist of documents, queries and relevance assessments. Obtaining consistent and exhaustive relevance assessments is crucial for the appropriate comparison of retrieval approaches. Whereas the evaluation methodology for flat text retrieval approaches is well established, the evaluation of XML retrieval approaches is a research issue. This is because XML documents are composed of nested components that cannot be considered independent in terms of relevance. This paper describes the methodology means that an XML retrieval system needs not only to find relevant information in the XML documents, but also to determine the appropriate level of granularity to return to the user. A consequence of this retrieval paradigm is that the relevance of an element is depends both on content and structural conditions. Evaluating the effectiveness of XML retrieval systems, hence, requires a test collection where the relevance assessments are provided according to a relevance criterion that takes into account the

PDF - PASCAL Members only - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Information Retrieval & Textual Information Access
ID Code:444
Deposited By:Patrick Gallinari
Deposited On:22 December 2004