Assessing academic language proficiency in university students: Developing and validating two new tools
Abstrakt
To gain knowledge about the research conducted in their area of study, university students need to be able to read articles published in international journals, which are predominantly written in English. The ability to understand academic English thus becomes a necessity.
Research shows that there are multiple factors contributing to successful comprehension of academic texts. These include possessing sufficient knowledge of academic vocabulary, as well as being able to decompose morphologically complex words, understand complex sentence structures, identify and track referents, understand and use academic discourse markers, organise and structure argumentative texts, determine the writer's stance, understand academic metalanguage and recognise and deploy the academic register (e.g.
Mancilla-Martinez & Lesaux, 2010; Uccelli et al., 2015). While tests assessing different dimensions of academic skills exist, they are either not freely available, suitable for university students, or designed to provide a comprehensive assessment of the different skills contributing to academic reading comprehension.
Yet, a complex diagnostic tool targeting such skills would benefit university students and ultimately their EAP and language teachers, as it would inform them about their students' strengths and weaknesses. Incorporating targeted activities in the instruction could then help the students become better academic readers.
This study involves the development of two computer-based instruments assessing the performance on different skills that contribute to successful comprehension of English academic texts. Both tests have automated scoring, making them suitable for diagnostic purposes.
The first instrument is a serial multiple-choice test based on Vocabulary Levels Test (Schmitt, Schmitt, & Clapham, 2001), which assesses the knowledge of word families sampled from the Academic Word List (Coxhead, 2000). The second tool, which was designed as a complement to the academic vocabulary test, is based on the Core Academic Language Skills Instrument (CALS-I) (Uccelli et al., 2015).
Since the CALS-I is suitable only for students up to middle school, the original tasks were modified and adapted for the university level. Both instruments were first piloted in August 2021 with students from several universities in the Czech Republic (n=46).
By May 2021, an expected total of 150 responses will be reached. The tools will subsequently be validated using a factor analysis approach and made accessible through an online platform developed for this purpose.
This presentation will describe the test development process and introduce the new platform. Coxhead, A. (2000).
A new academic word list. TESOL quarterly, 34(2), 213-238.
Mancilla-Martinez, J., & Lesaux, N. K. (2010).
Predictors of reading comprehension for struggling readers: The case of Spanish-speaking language minority learners. Journal of Educational Psychology, 102(3), 701-711.
Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test.
Language Testing, 18(1), 55-88. Uccelli, P., Barr, C.
D., Dobbs, C. L., Galloway, E.
P., Meneses, A., & Sánchez, E. (2015). Core academic language skills: An expanded operational construct and a novel instrument to chart school-relevant language proficiency in preadolescent and adolescent learners.
Applied Psycholinguistics, 36(5), 1077-1109.