PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Constructing a Textual Semantic Relation Corpus Using a Discourse Treebank
Rui Wang and Caroline Sporleder
In: LREC 2010, 19-21 May 2010, Valetta, Malta.


In this paper, we present our work on constructing a textual semantic relation corpus by making use of an existing treebank annotated with discourse relations. We extract adjacent text span pairs and group them into six categories according to the different discourse relations between them. After that, we present the details of our annotation scheme, which includes six textual semantic relations, backward entailment, forward entailment, equality, contradiction, overlapping, and independent. We also discuss some ambiguous examples to show the difficulty of such annotation task, which cannot be easily done by an automatic mapping between discourse relations and semantic relations. We have two annotators and each of them performs the task twice. The basic statistics on the constructed corpus looks promising: we achieve 81.17% of agreement on the six semantic relation annotation with a .718 kappa score, and it increases to 91.21% if we collapse the last two labels with a .775 kappa score.

EPrint Type:Conference or Workshop Item (Paper)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Natural Language Processing
ID Code:7122
Deposited By:Caroline Sporleder
Deposited On:04 March 2011