Pissarro Sounds

In the exhibition "Camille Pissarro. The Studio of Modernism", a distinctive sound art experience called "Pissarro Sounds" offered a sensory exploration of the artist's life and work. With the help of binaural spatial sonics and precise location tracking, visitors were immersed in three-dimensional sound spaces generated based on their position in the space. This installation, commissioned by the museum, was created by the artist Moritz Fehr. It is the first public application of the Innosuisse research project "Immersive Audio Guiding System" (IAGS), a collaboration of the FHNW School of Music, iart and Idee und Klang Audio Design.

"Pissarro Sounds" spanned five rooms of the exhibition. With each space, visitors explored a dynamic soundscape embedded with Pissarro's original texts and ambient sonic textures. A special feature of the project: many of the recordings were taken from the actual site of Pissarro’s daily life. For example, visitors experienced the sonic environment of his studio, a converted barn that still stands in Eragny, surrounded by the family garden.

"Pissarro Sounds" expanded the perception of the real exhibition space with live-generated soundscapes. Based on an interactive 3D model, an exact replica of the exhibition space, it is comparable to a computer game consisting of virtual rooms, various objects, sounds, interactions and trigger zones. This "digital twin" concept is the core of the Mixed Reality Platform (MxRP) developed by iart especially for museum application.

High-precision location tracking records the movement of visitors in real time and transmits their position to the 3D model. The tracking is based on ultra-wideband technology (UWB), which is particularly suited to real-time localisation indoors. It is carried out via special headphones, which in terms of appearance and handling, are indistinguishable from conventional headphones, but in which additional technology is integrated. Each headphone receives signals from UWB transmitters distributed in the exhibition space, and can determine its own position with an accuracy of twenty centimetres. An IMU sensor (Inertial Measurement Unit) also determines the head orientation of the visitors. In this way, the system not only senses the users’ position, but also the direction they are looking in.

All this information is processed in real time. In the so-called binaural synthesis, both the properties of the human ear and spatial acoustics are taken into account: How big is the room? Are we indoors or outdoors? What material are the walls made of? Where is the sound source located in relation to the user? The answers to these questions gave the experience of Pissarro's studio in Eragny a deceptively real three-dimensionality, in which visitors could hear the voice of a virtual Pissarro wandering through the space.

iart's Mixed Reality Platform enables future-oriented applications for museums and exhibitions on the basis of a digital twin. Apart from the current application, it also offers, for example, an ideal groundwork for the implementation of visual augmented realities, the provision of context-driven additional information in the language of the respective user or automatic light, sound, and video triggered by the position of the visitor.