SMPC 2022 (Portland). Cognitive Musicology with AR/VR: Visualisation of Musical Narrative/Structures

34 Просмотры

Спасибо! Поделитесь с друзьями!

Вам не понравилось видео. Спасибо за то что поделились своим мнением!

Издатель ноя 22, 2022

Authors: Svetlana Rudenko, Maura McDonnell, Timothy Layden and Mads Haahr
Cognitive Musicology refers to the study of musical thinking. Musical thinking is a complex involving memory, emotion, culture, metaphorical thinking, musical language and cross-modal associations. Bulat Galeyev stated that ‘Synesthesia calls forth such notions as “melody line,” “the hearing space” and “tone color,” and makes it possible to perceive sounds and chords as “sharp,” “dull” or “high.” Synesthesia (and the particular case of “color hearing”) is the essential component of musical thinking, first of all, in music intended to evoke images.’ (Galeyev, 2007) Essentially, perception of music is always an embodied multisensory experience. Cognitive Musicology is a relatively new field that uses computational tools to reflect on musical language and the same time, to understand music consciousness and perception. In this presentation, we explore what technology such as Augmented and Virtual Reality (AR and VR) can offer for visualisation of classical music: to reflect on musical structures and music narrative. We discuss three cases: (1) First Augmented Reality Classical Concert is an AR/MR experience developed in 2017 and based on Gustav Holst’s “The Planets” Op. 32; (2) Intermezzo Brahms Op. 117 N2 with AR based on synaesthesia art; and (3) Scriabin’s Sonata N5 Op. 53 with visuals reflecting on archetypes of musical texture and music narrative as a prototype for a VR experience. The third case shows the design reflecting on structural components (such as archetypes of musical texture), cultural (images of epoch and life composer) and associative (subjective cross-modal associations experience of the performer), and synaesthesia art, reflecting on sensory perception. https://vimeo.com/382956724
In conclusion, applications in AR/VR for classical music could be important tools for music education, stimulating associative thinking and creativity.

Keywords: Cognitive Musicology, Interactive narrative, AR/ VR, Music narrative, Synaesthesia art.