Annotators’ Certainty and Disagreements in Coreference and Bridging Annotation in Prague Dependency Treebank
2013
Abstract
We present the results of the coreference and bridging annotation in the Prague Dependency Treebank 2.0. The annotation is carried out on dependency trees (on the tectogrammatical layer).
We describe the inter-annotator agreement measurement, classify and analyse the most common types of annotators’ disagreement. On two selected long texts, we asked the annotators to mark the degree of certainty they have in some most problematic points; we compare the results to the inter-annotator agreement measurement.