PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Summarization system evaluation revisited: N-gram graphs
George Giannakopoulos, Vangelis Karkaletsis, George Vouros and Panagiotis Stamatopoulos
ACM Transactions on Speech and Language Processing (TSLP) Volume 5, Number 3, pp. 1-39, 2008. ISSN 1550-4875


This article presents a novel automatic method (AutoSummENG) for the evaluation of summarization systems, based on comparing the character n-gram graphs representation of the extracted summaries and a number of model summaries. The presented approach is language neutral, due to its statistical nature, and appears to hold a level of evaluation performance that matches and even exceeds other contemporary evaluation methods. Within this study, we measure the effectiveness of different representation methods, namely, word and character n-gram graph and histogram, different n-gram neighborhood indication methods as well as different comparison methods between the supplied representations. A theory for the a priori determination of the methods' parameters along with supporting experiments concludes the study to provide a complete alternative to existing methods concerning the automatic summary system evaluation process.

EPrint Type:Article
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Natural Language Processing
ID Code:4453
Deposited By:George Giannakopoulos
Deposited On:13 March 2009