Humanist evaluation methods in locative media design

Abstract

Media design can be used for research purposes if it includes a clearly defined research question, and clear evaluation to see whether an answer to the research question has been found. Using a project with locative media for classical music communication as our example, we discuss common evaluation methods from the User Experience field, observing that they all tend to test “interface” and not “content.” Instead we propose three other methods of evaluation, that have a basis in humanist theories, such as textual analysis and genre studies: (1) Qualitative interviews with evaluators after the evaluation, asking them to describe the service in their own words, followed by a semantic analysis to get at how they have understood the service. (2) Within-subject A/B tests with alternative versions that are different in key aspects. (3) Peer review by experienced design researchers, who are likely to have a more fine-tuned vocabulary to express their opinions.