Facing the Gaze

Commissioned by Park Avenue Armory in New York City as a site-specific work, Hansel & Gretel is a vast interactive work, transforming the Park Avenue Armory into a dystopian public park. The project was conceived by Ai Weiwei, Jacques Herzog and Pierre de Meuron and curated by Tom Eccles and Hans Ulrich Obrist. iart helped Herzog & de Meuron develop the media installation for the hall and was responsible for implementing and integrating the tracking technology. The Park Avenue Armory is a former military facility. It's drill hall is a 55,000 square foot space, with 80-foot high barrel vaulted ceilings.

Hansel & Gretel oscillates between architecture and art as it explores the perception of space and the status of public space in the context of ubiquitous surveillance. The title refers to the fairy tale by the Grimm Brothers in which a brother and sister get lost in the forest - a place that is familiar to them. The entrance to the installation is an inconspicuous side door from which a narrow darkening tunnel guides the visitor to the vast and nearly completely dark Drill Hall. Drones whir over the heads of visitors, following and transmitting their movements to large screens in the Armory Head House.

All the visitors are recorded by a large, decentralized, interactive surveillance system, which iart helped conceive, develop, and implement. Fifty-six projectors, each with its own iart tracking unit, follow visitors around the hall, recording their movements, converting its recordings using specially developed software, and transmitting it back to the projectors. Images of the visitors from above are projected onto the sloped floor in real-time while their movements are followed by red rectangles. While wandering through the hall, visitors leave a track of orthogonal white lines behind – almost like a digital equivalent of Hansel and Gretel's breadcrumbs. The viewers become an integral part of the work by means of their interactions with the surveillance system: they counter the initially menacing environment with playful attempts to use their digital shadows as a design element: the selfie in the surveilled space as a way to win back one's own likeness.

The projection area measures 12,360 × 7,850 pixels; each of the fifty-six projectors works independently in a decentralized network in order to maximize flexibility. Infrared emitters ensure that the infrared cameras can record images of the visitors in the nearly completely dark hall. Over several iterations, an adaptive software was developed that can record the people, follow their movements, and finally project their 'digital shadows.'