The chapter focuses on the issue of (in)comparability of student data based on self-assessment, focusing on the area of measuring student ICT knowledge and skills. In educational research, questionnaires containing self-report items with rating scales are widely used to measure various concepts (e.g. willingness to pay attention to the subject, anxiety about the subject, motivation to learn the subject). On the basis of such data, different groups of pupils are then compared to each other (according to the type of school, etc.). However, the data may be skewed by how different groups of students use rating scales and report the level of concepts measured. This can lead to erroneous conclusions about the level of these concepts in different groups of students. Methodological limitation can have fundamental impacts on the informative value of the results of pedagogical research and, by extension, research in other social sciences. On the basis of inaccurate results, inappropriate measures in the field of educational policy may subsequently be adopted, which may negatively affect further development in the field of education.
The authors dealt with the issue in the context of measuring the level of ICT knowledge and skills of pupils in the Czech Republic (GAČR 17-02993S - Factors influencing the self-assessment of ICT skills of secondary school students). In accordance with the purpose of the book, in this chapter we present the procedures and challenges of the research project under consideration with an emphasis on methodological aspects. We highlight recommendations for further research and overlaps of the research carried out, especially in the methodology of research in the social sciences.