Abstrakt
One of the many aspects of spoken learner language touched upon in the Common European Framework of Reference (CEFR) (Council of Europe, 2001; 2018) is accuracy. According to the CEFR, B2 speakers are characterized by "a relatively high degree of grammatical control" and no longer produce errors which cause misunderstandings.
C1 speakers are characterized by "a high degree of grammatical accuracy" with rare, difficult to spot, and generally corrected errors. As this gives only a very general idea about accuracy at these levels and as only grammatical accuracy is mentioned, the present study aims to shed more light on the differences between B2 and C1 spoken learner English as regards various facets of accuracy.
We analyse the frequency of errors in several categories (e.g. grammatical, lexical, lexico-grammatical), identify which types of errors characterize each level, and which errors disappear but also persist at level C1. We also examine whether task design impacts accuracy, and whether speakers with two typologically different L1s, Czech and Taiwanese, appear to follow similar developmental patterns.
This research is unique in that it exploits corpora which have been post-hoc assessed for proficiency and thus provides an empirical basis for the understanding of CEFR B2 and C1 accuracy, and identifies particular areas of difficulty for speakers at these levels. The data derives from the Czech (Gráf, 2015a) and Taiwanese (Huang, 2014) components of the spoken learner English corpus LINDSEI.
These two subcorpora have been rated for accuracy by professional IELTS examiners who also received a special CEFR rater-standardisation training (Huang et al., 2018). The rating of our data resulted in the identification of 9 B1, 52 B2, 37 C1 and 2 C2 speakers.
The B1 and C2 speakers were excluded from the analysis. The transcriptions of the remaining 89 speakers' 15-minute interviews (see Table 1 for the size of the dataset) were error-tagged following the Louvain error-tagging manual (Dagneaux et al., 2008) extended by Gráf (2015) to include 59 error types at grammatical, lexical, lexico-grammatical, and syntactical levels.
B1 (n) tokens B2 (n) Tokens C1 (n) tokens C2 (n) tokens Total tokens Czech 0 0 13 24,165 35 66,305 2 5,499 95,969 Taiwanese 9 10,028 39 55,707 2 3,785 0 0 69,520 Total 9 10,028 52 79,872 37 70,090 2 5,499 16,5489 Table 1. Size of the two subcorpora (LINDSEI_CZ, LINDSEI_TW): numbers of speakers and numbers of tokens the speakers at different accuracy levels produced.
The error analysis revealed a total of 5,108 errors. The comparison of error rates (numbers of errors per hundred words, henceforth phw) between the two levels of proficiency showed that B2 speakers produce errors at a higher frequency (6.7 errors phw at B2 and 1.9 errors phw at C1), and this was similar for the performance in the three different tasks (monologue, dialogue and picture description).
The Louvain taxonomy enabled the categorization of errors and the results reveal that grammatical errors are the most frequent, followed by errors of lexical nature. Lexico-grammatical and other types are much less frequent (see Table 2).
Morphological errors Grammar errors Lexical errors Lexico-grammatical errors Word order errors Infelicities B2 7 (0.2%) 3,023 (72.5%) 701 (16.8%) 152 (3.7%) 196 (4.7%) 76 (1.8%) C1 3 (0.3%) 558 (59.4%) 277 (29.5%) 55 (5.9%) 34 (3.6%) 13 (1.4%) Table 2. Representation of error types at B2 and C1 levels.
At the grammatical level, article errors and verb tense errors are the most frequent for both groups of speakers. At C1, these are less frequent and also some types of errors common at B2 do not appear here (e.g. errors in the use of reflexive pronouns).
Persistent errors have been identified especially in the use of articles, the use of the present perfect, and in tense agreements. Differences also occur between Czech and Taiwanese speakers.
The latter produce errors at a higher frequency and are more prone to commit errors in areas which the English Vocabulary Profile classifies as B1 (use of past simple). Lexical errors involve mostly those affecting single lexemes (especially prepositions).
Qualitative analysis of a selection of these errors reveals that none of these errors impact intelligibility. Whilst the CEFR claims that at C1 errors are generally corrected, this does not appear to be the case in our dataset: most of the errors still seem to be errors of competence rather than performance and the speakers do not appear to be aware of them.
Future analysis will focus in more detail on the detailed inspection of these errors which will identify with even greater precision areas of potential improvement and thus a target for pedagogical intervention. Besides, our corpora have now been supplemented by approximately 80 more recordings which include A2 and B1 speakers and thus we will be able to include these levels in our comparison.