Abstract
This paper presents the results of the WMT14 Metrics Shared Task. We asked participants of this task to score the outputs of the MT systems involved in WMT14 Shared Translation Task.
We col- lected scores of 23 metrics from 12 re- search groups. In addition to that we com- puted scores of 6 standard metrics (BLEU, NIST, WER, PER, TER and CDER) as baselines.
The collected scores were eval- uated in terms of system level correlation (how well each metric’s scores correlate with WMT14 official manual ranking of systems) and in terms of segment level correlation (how often a metric agrees with humans in comparing two translations of a particular sentence).