Sound Spatialization

Project: Compositional Applications of Auditory Scene Synthesis in Concert Spaces via Gestural Control
Participants: (IDMIL): Mark T. Marshall, Joseph Malloch, Marcelo M. Wanderley; (MPCL/RTML): Nils Peters, Georgios Marentakis, Stephen McAdams; (DCS): Sean Ferguson; (RPI): Jonas Braasch
Time period: 2005–2008

This project aims to develop novel compositional and technological methods for the advanced use of the multidimensional nature of auditory space in music composition. Specifically, the project involves the examination, design and construction of novel methods for gesture control of sound source position, sound source parameters and room model parameters within a sound spatialization system.

'Cellist Chloé Dominguez rehearsing with motion sensors attached to her arms.

‘Cellist Chloé Dominguez rehearsing with motion sensors attached to her arms.


Research being performed within this project includes:

  • Motion capture of performer gestures to determine which aspects of the gestures may be used to control spatialization
  • Development of novel gestural controllers for manipulation of sound source and room model parameters
  • Design of systems to unobtrusively capture ‘accompanist gestures’ which performers make during performance
  • Evaluation of existing controllers as tools for gesture controlled spatialization
  • Examination of novel methods and metaphors for manipulating sound sources in a spatialization system

Other aspects of the project which are being developed by our research partners include:

  • The design and development of a 24-channel audio virtual environment (AVE)
  • Examination of the perception of spatial audio
  • Compositional aspects and uses of gesture-controlled spatialization

Along with the research aspects of this project, there is a significant musical component. A number of pieces are being written, including works both for solo cello and also for a larger ensemble. The first of these performances took place as part of the 4th International Conference on Enactive Interfaces, and will be followed by larger performances in Montreal, QC, Canada as part of the MusiMarch08 event and again later in 2008 in BC, Canada.

Related publications:

Georgios Marentakis, Joseph Malloch, Nils Peters, Mark T. Marshall, Marcelo M. Wanderley, and Stephen McAdams. “Influence of performance gestures on the identification of spatial sound trajectories in a concert hall”. In Proceedings of the The 14th International Conference on Auditory Display (ICAD 08), Paris, France, 2008.

Mark T. Marshall, Joseph Malloch, and Marcelo M. Wanderley. “Non-conscious control of sound spatialization”. In Proceedings of the 4th International Conference on Enactive Interfaces, Grenoble, France, pp. 377–380, 2007.

Mark T. Marshall, Joseph Malloch and Marcelo M. Wanderley. “A Framework for Gesture Control of Spatialization.” Presented at the 2007 International Gesture Workshop, Lisbon, Portugal, 2007.