Controlling or Being Controlled? Exploring Embodiment, Agency and Artificial Intelligence in Interactive Music Performance

Abstract

How can we make music with artificial intelligence (AI) in the future? Unlike most studies on AI and music, this dissertation focuses on physical interaction and the ways in which the computer can respond to body movement. Based on experimental music practices, it argues that diversifying artistic repertoires in music-making is crucial for the future of music. Emphasis has been placed on realizing creative works and their evaluations in ecological environments. The exploration starts from an extensive literature review that sketches a broad picture of alternative control paradigms in the performing arts, different types of musical AI, and embodied approaches to human cognition. Then follows a methodological presentation and discussion structured around the four projects that the dissertation is focused on. The shared music–dance piece Vrengt demonstrates the musical possibilities of sonic microinteraction and provides a conceptual model of co-performance. The muscle-based instrument RAW implements various AI techniques to explore a chaotic instrumental behavior and automated interaction with an improvisation ensemble. A novel empirical study sheds light on how guitar players transform biomechanical energy into sound. The collected multimodal dataset is used as part of a modeling framework for “air performance.” The coadaptive audiovisual instrument CAVI uses generative modeling to automate live sound processing and investigates expert improvisers’ varying sense of agency. All in all, this dissertation stresses the importance of embodied perspectives when developing musical AI systems. It emphasizes an entwined artistic–scientific research model for interdisciplinary studies on performing arts, AI, and embodied music cognition.