PASCAL - Pattern Analysis, Statistical Modelling and Computational Learning

Findings of the 2011 Workshop on Statistical Machine Translation
Chris Callsion-Burch, Philipp Koehn, Christof Monz and Omar Zaidan
In: Sixth Workshop on Statistical Machine Translation, 7-10 July 2011, Edinburgh.


This paper presents the results of the WMT11 shared tasks, which included a translation task, a system combination task, and a task for machine translation evaluation metrics. We conducted a large-scale manual evaluation of 148 machine translation systems and 41 sys- tem combination entries. We used the rank- ing of these systems to measure how strongly automatic metrics correlate with human judg- ments of translation quality for 21 evaluation metrics. This year featured a Haitian Creole to English task translating SMS messages sent to an emergency response service in the af- termath of the Haitian earthquake. We also conducted a pilot ‘tunable metrics’ task to test whether optimizing a fixed system to differ- ent metrics would result in perceptibly differ- ent translation quality.

PDF - Requires Adobe Acrobat Reader or other PDF viewer.
EPrint Type:Conference or Workshop Item (Oral)
Project Keyword:Project Keyword UNSPECIFIED
Subjects:Natural Language Processing
ID Code:9340
Deposited By:Christof Monz
Deposited On:16 March 2012