Shortening the Test? A Comparison of Test Shortening Strategies

Abstract

Sesjon is a general entrance exam to the Norwegian Armed Forces. Due to time limitations, it would be necessary to shorten some parts of sesjon to add new assessments. This study investigates whether it is feasible to shorten the nonverbal intelligence test, a cognitive figures test, at sesjon with a computerized adaptive test (CAT), and explores whether simpler and more cost-effective short forms could shorten the test as effectively as the CAT. These questions were investigated with a simulation study, that simulated both new test-takers and them answering an item bank based on the cognitive figures test, a CAT, and the other short forms. The test forms were compared at how well they could estimate the simulated test-takers' ability level. The results showed that the CAT was almost as precise at estimating test-takers' ability level as the item bank when considering that the CAT administered fewer items and that an item exposure short form, a short form which used the most items administered in the CAT, was almost as good at estimating the test-takers' ability level as the CAT. An important finding was that the cognitive figures test had too many easy items and very few difficult items. Due to this, the recommendation is that the CAT or the item exposure short form could be used as screening tools instead of the cognitive figures test and that none of the tests should be used for measuring nonverbal intelligence unless more difficult items are added to the cognitive figures test.