Cross-modal correspondences: different modes, common codes? Investigating musical engagement with an Ecological cognitive approach.

Abstract

This project uses music to explore the way in which the environment affects cross-modal correspondences (CMC). This phenomenon relates to the way senses might be synchronised. Gibson’s ecological theory of perception and the subsidiary theory of 4E cognition provide the theoretical starting points for the research. The thesis builds on data collected in relation to the public research concert MusicLab 8: Synaesthesia. The thesis consists of a two-stage experiment, one representing the live performance and another a home listening context. Data collection was done using open-ended surveys, interviews and audio-visual recordings. The data is studied using qualitative methods such as content analysis and coding. The significance of the work contributes to the multisensory experience in music and HCI research. The exploration of a 4E cognition framework opens up interesting possibilities to reconsider the generative role of musical engagement. With this in mind, it may provide practitioners with a more solid, holistic foundation when planning and designing musical events where cross-modalities are more important (i.e. audiences’ participation, staging, sound array positioning, visualisation).