Possibilities of Practical Work with Data from International Large Scale Educational Assessments: Problems and Practical Solutions
Abstract
The Czech Republic participated in the last 20 years in several international large scale educational assessment, especially TIMSS, PIRLS, PISA, ICCS, PIAAC and ICILS. Data for all these surveys are always publicly available after completion of each cycle (Soukup, 2012).
However, Czech educational researchers used data very rarely (mostly only commented published results), although this data is very rich, and allow the use of complex statistical approaches. Problems when working with these data arise mainly from the following factors: (a) The data include weights, which take into account unequal probabilities of selection for individual respondents, and these weights should be used to correct estimates of population parameters; (b) the data come from a multistage sampling, and it is necessary to modify the standard error estimates for the calculation of statistical tests or confidence intervals (the use replication approaches, e.g.
Jackknife); (c) the multiple data files at national and international level (i.e. there are actually dozens of datasets and their merge are not entirely trivial); (d) some variables not measured directly (typically results in cognitive tests) and should be considered a measurement error of these latent variables (IRT methodology and plausible values can be used). The aim of the text will be to serve several examples of analytical procedures to demonstrate the proper procedures for working with data from an international large scale educational assessment and also to provide an overview of available software for processing such data so that Czech educational researchers can routinely use