ERCIM News No.37 - April 1999

The Computer makes Music in Pisa

by Leonello Tarabella

---

The keywords characterising the activity of the Computer Music laboratory of CNUCE-CNR, Pisa, are man-machine interaction and real time gesture control. The computer interacts with the artist to produce music in many different ways. In order to achieve this, a special language called Real-Time Concurrent Pascal Music (RTCPM) has been implemented. Using this language, a musical performance can be defined in terms of many procedures running simultaneously and interacting with the composition program during execution. As an evolution of RTCPM, a new language named GALileo (Graphic/ALgorithmic) which includes both visual programming and signal processing “facilities” is now under development.

In order to create and control interactive live computer music/graphics performances, the CNUCE Computer Music lab employs infra-red beams and video captured image processing technologies for the remote sensing of moving objects handled by performers or of gestures made by them. The most important devices that have been developed using this approach are:

• TwinTowers (in collaboration with IEI-CNR) - this system receives information from the movement of the performer’s hands. It consists of two sets of four infra-red sensing devices that create two zones of the space (the vertical edges of two square-based parallelpipedon, or towers) in which an object can be detected in terms of distance and front and side rotations with respect to a reference frame. The system thus implements a kind of double aerial tri-dimensional joy-stick.
• Aerial-Painting-Hands - this system, developed by Giuseppe Scapellato and normally used by the artist Marco Cardini (see Ercim News No.35), detects the positions and movements of a performer’s hands; the performer wears a pair of different coloured gloves and moves his/her hands in a CCD camera video range. The signal is sent to a computer which processes the images and recognises the x-y positions of both hands and the open/close status of the right hand. The computer then synthesizes in real-time coloured images projected on a large video screen according to the movements of the performer’s hands. While painting in the air, the performer is also creating computer generated music in real-time An advanced version of this system is now under development. The new system will recognise the posture, rotation and position in space of the hands, enabling the transmission of more sophisticated information and thus a greater expressiveness during the perfomance.
• UV-stick - this system consists of a short straight stick (lit by a UV lamp) and moved in the visual range of a CCD camera connected to a computer which recognises the 2-D position and 3-D rotations of the stick. The data acquired is used to manage sound synthesis algorithms and create music in real-time.
• Imaginary Piano - where the hands of a performer play in the air with no real keyboard in front of a camera, connected to a video digitizer card. A special application running on the computer recognizes the player’s gestures, interacts with a RTCPM program and executes an external sampled piano sound.

The Computer Music Laboratory also participated in the EC ESPRIT project CATS (Computer Aided Theatrical Score) for theatre and cinema direction simulation.

For more information, see: http://spcons.cnuce.cnr.it/music/cmd.html

Please contact:

Leonello Tarabella - CNUCE-CNR
Tel: +39 050 593 276
E-mail: l.tarabella@cnuce.cnr.it