Publication at Faculty of Mathematics and Physics |
2010
Abstract
We present several ways of measuring the inter-annotator agreement in the ongoing annotation of semantic inter-sentential discourse relations in the Prague Dependency Treebank (PDT). Two ways have been employed to overcome limitations of measuring the agreement on the exact location of the start/end points of the relations.
Both methods – skipping one tree level in the start/end nodes, and the connective-based measure – are focused on a recognition of the existence and of the type of the relations, rather than on fixing the exact positions of the start/end points of the connecting arrows.