Introduction

future-instruments.net

Introduction

Why do we need new musical instruments and interfaces?

With the rapid development of computers during the last two decades, many new sound generation methods have appeared to the market. Composing tools have radically evolved as well, with the use of sequencers and multi-track recording systems. Despite of this, gestural interfaces have not evolved very much. They are still based on the model of classical instruments, the keyboard being the most widely used. However, there is a real need for new musical interfaces. The gestures required to play classical instruments are bound to the physics of the sound production and to the mechanics of the instrument. They make sense in this particular context. With new sound generation methods and new composition parameters, such as the variation of a sound texture in time, for instance, those gestures may not be adapted anymore. As a consequence, it is necessary to find new musical gestures that are closely connected with the inner movements of music, as they are created with modern compositional means. New musical interfaces capable of interpreting today’s music are required, with new gestural and functional capabilities, but with the same level of expressiveness found in traditional musical instruments.

What is our line of research?

There is an ever growing community of artists and scientists doing research and experiments in this field, and the annual conference NIME (New Interfaces for Musical Expression) reflects very well the growing interest for this topic. However, since the invention of the Theremin in the 20’s - the first known electronic instrument with a gesture interface that differed radically compared to classical instruments - most of the developments tended to separate the instrument in two parts:

- The gesture interface (the controller)

- The sound generator (usually the computer or a sound module)

This separation has been even reinforced by the invention of MIDI, which brought a standard protocol of communication between those two parts. On the contrary, with classical instruments, the performer interacts closely and most of the time directly with the source vibration (ex guitar, violin, etc). This is due to the symbiotic nature of the interface, which features a close integration of sound generation and control, and results in highly expressive capabilities. Our goal is to create new musical instruments that keep the original essence of classical ones, but with new gesture interactions and extended control capabilities. Therefore, we look for ways to re-unify the sound generation and the control over the sound in the same interface. One common approach is to augment an acoustic instrument by fitting extra sensors, such as pressure sensors, ribbon controllers, switches, etc. Our preferred approach is to take simple vibrating elements, such as wood boards or metallic plates and try to make them interactive. Our vision is to create hybrid electronic-acoustic instruments that are performed using the vibrating body as a controller. The idea behind this is to pick-up the vibration generated by the interaction with an object to use it as a sound source and to employ at the same time some of the interaction parameters, such as the position of the contact point(s) for instance, to control the processing of the sound by means of a computer or effect processor.

In order to achieve this, we have to look for interactive technologies that are the less intrusive: the sensing methods must not interfere with the sound source and thus let the object vibrate freely. This led us to consider primarily acoustic sensing technologies (see the TAI-CHI project) and, more recently, computer vision technologies. Acoustic sensing has the advantage of integrating smoothly with an object and with a minimum of intervention (only a few sensors to glue on the surface), but are still limited to non-reverberant materials, which are thus not very interesting as a sound source. An example of this is our Percussion Tray demo, which consists of a plastic tray transformed into a drum controller. In this case, the vibration of the object is only used to locate the point of contact but not as a sound source. On the other hand, computer vision techniques allow for using any kind of material but make difficult to detect when touching or not an object. We have tried to override this problem by combining acoustic and vision technologies, as illustrated by the Touch Table project. Currently, we are developing a new technique that allows multi-touch detection on any kind of flat surface, including non transparent one (more on this soon).

Alain Crevoisier