Cross-cultural (in)comparability of students' self-reports in (inter)national large-scale surveys like the PISA

Abstract

Self-reported data are largely used in educational sciences to compare different groups of students. In questionnaire surveys, students are, for example, asked to evaluate their own motivation for learning, attitudes towards school, or satisfaction with life.

However, current research suggests that the comparability of self-reports among different groups of students may be hindered by the differences in scale usage. Students from different cultures, countries, schools, and with different socio-economic background could use the scale categories in self-reported questions differently: two groups of students with the same actual level of motivation for learning can report their motivation differently, students in one group can report high levels of motivation, while the students in the other group report low levels of motivation.

There are several innovative methods which help to identify the heterogeneity in reporting behavior (differences in scale usage, DIF) among different groups of students and subsequently adjust their self-reports for this heterogeneity. In this invited lecture, we will introduce these methods and show their use in (inter)national large scale surveys like the Programme for International Student Assessement (PISA).