Identifying Culturally Preferred Response Patterns Using the Anchoring Vignette Method and the Overclaiming Technique
Abstract
Self-reported data are commonly used in large-scale surveys to compare students' non-cognitive skills, attitudes, opinions, values and also self-reported knowledge among different countries and different socio-economic groups. International large scale surveys like the PISA study include self-reported questions in questionnaires as well.
Examples of such questions are "I want to be one of the best students in my class" with a four-point Likert scale ranging from "strongly agree" to "strongly disagree" (Achievement motivation, PISA 2015 questionnaire), "I get very tense when I study for a test" with the same Likert scale (Test Anxiety, PISA 2015 questionnaire), or "Thinking about mathematical concepts: how familiar are you with the following terms? Exponential function, divisor ..." with a scale ranging from "never heard of it" to "know it well, understand the concept". However, in the literature, there is a concern that the response patterns may hinder the comparison of self-reported data among different groups of students.
In this presentation, we introduce two methodological approaches: the anchoring vignette method and the overclaiming technique. Using these approaches we identify culturally preferred response patterns and use these approaches to enhance the comparability of self-reported data.
We specifically analyze PISA 2012 and PISA 2015 datasets where both approaches were implemented.