Charles Explorer logo
🇨🇿

Modelling crosslinguistic n-gram correspondence in typologically different languages

Publikace na Filozofická fakulta |
2021

Tento text není v aktuálním jazyce dostupný. Zobrazuje se verze "en".Abstrakt

N-gram analysis (popularized e.g. by Biber et al., 1999) has become a popular method for the identification of recurrent language patterns. Although the extraction of n-grams from a corpus may seem straightforward, it proves to be very challenging when applied cross-linguistically (cf. e.g.

Ebeling and Ebeling, 2013; Granger and Lefer, 2013; Cermakova and Chlumska, 2017). The major issue is that the quantities of n-grams of a certain length in typologically different languages do not correspond.

Consequently, n-grams of a given length may function differently across languages, rendering a direct comparison inadequate. Our paper introduces a function capable of modelling the relation between the quantities of n-grams in typologically distant languages, using the example of Czech and English (and some other language pairs).

Based on our model, we can suggest what n-gram lengths should be contrasted to better reflect the size of n-gram inventories in each language. The correspondence may not be intuitive (e.g. a Czech 2-gram may best correspond to an English 2.5-gram), but it still provides researchers with a general guide as to what might be useful to include in their analysis (e.g. in this case 2-grams in Czech and 2- and 3-grams in English).