The effectiveness of novice users in usability testing

Abstract

Current research is fine-tuning the usability evaluation methods in order to aid current and future usability practitioners. Some of the main research topics are sample sizes and appropriate statistical methods for analysing usability test data. However, few studies have focused on participants with different experience levels on test samples.

This thesis aims to fill this gap and add knowledge to the ongoing research on users with different experience levels in usability testing. The purpose of this study is to investigate the effectiveness of using novice- and expert users in usability tests.

23 individuals participated in a series of usability tests. The participants were grouped into novice and expert users. The results from the tests were analysed and compared in order to find out if novice users encountered the same type of usability problems as expert users. An additional analysis was performed to study if novices provided a larger number of problems than expert users.

The analysis suggests that novice users are likely to reveal the same type of usability problems as expert users in usability tests. Furthermore, the analysis suggests that novice users are likely to uncover a larger number of usability problems than expert users.

These findings implicate both the researchers’ and usability practitioners’ community. First of all, this thesis supports previous research on similar topics. Secondly, usability practitioners will benefit from this knowledge during the planning and recruitment process of usability tests.