Explanatory item response modelling of an abstract reasoning assessment: A case for modern test design

Abstract

Assessment is an integral part of society and education, and for this reason it is important to know what you measure. This thesis is about explanatory item response modelling of an abstract reasoning assessment, with the objective to create a modern test design framework for automatic generation of valid and precalibrated items of abstract reasoning. Modern test design aims to strengthen the connections between the different components of a test, with a stress on strong theory, systematic item design, and advanced technological and statistical tools. Such an approach seeks to improve upon the traditionally weak measures found in education and social sciences in general.

The thesis is structured in two parts. Part one presents the theoretical basis of the dissertation, and part two presents the empirical analysis and results of the assessment. The first chapter establishes an understanding of the general field of which this study has been conducted. The second chapter delves into the particular content domain relevant for the assessment. The third chapter presents the actual assessment that is the object of investigation. The fourth chapter presents a comprehensive report on a cognitive lab. The fifth chapter presents the factors on which the actual explanatory item response modelling of the assessment is founded on. The last chapter present a general discussion and conclusion of the study.