Facing the Gaze

The Installation by Herzog & de Meuron and Ai Weiwei Studio in the Park Avenue Armory Made Surveillance Visible



Commissioned by Park Avenue Armory in New York City as a site-specific work, Hansel & Gretel is a vast interactive work, transforming the Park Avenue Armory into a dystopian public park. The project was conceived by Ai Weiwei, Jacques Herzog and Pierre de Meuron and curated by Tom Eccles and Hans Ulrich Obrist. iart helped Herzog & de Meuron develop the media installation for the hall and was responsible for implementing and integrating the tracking technology. The Park Avenue Armory is a former military facility. It's drill hall is a 55,000 square foot space, with 80-foot high barrel vaulted ceilings.

What's it about

Hansel & Gretel oscillates between architecture and art as it explores the perception of space and the status of public space in the context of ubiquitous surveillance. The title refers to the fairy tale by the Grimm Brothers in which a brother and sister get lost in the forest - a place that is familiar to them. The entrance to the installation is an inconspicuous side door from which a narrow darkening tunnel guides the visitor to the vast and nearly completely dark Drill Hall. Drones whir over the heads of visitors, following and transmitting their movements to large screens in the Armory Head House.


  • Wade Thompson Drill Hall: 55.000 ft²
  • Drones fly above the visitor's heads
  • Surveillance images in real-time
  • 56 beamers project them back onto the ground
  • Visitors can play with their digital imprints

A Decentralized Surveillance System

All the visitors are recorded by a large, decentralized, interactive surveillance system, which iart helped conceive, develop, and implement. Fifty-six projectors, each with its own iart tracking unit, follow visitors around the hall, recording their movements, converting its recordings using specially developed software, and transmitting it back to the projectors. Images of the visitors from above are projected onto the sloped floor in real-time while their movements are followed by red rectangles. While wandering through the hall, visitors leave a track of orthogonal white lines behind – almost like a digital equivalent of Hansel and Gretel's breadcrumbs. The viewers become an integral part of the work by means of their interactions with the surveillance system: they counter the initially menacing environment with playful attempts to use their digital shadows as a design element: the selfie in the surveilled space as a way to win back one's own likeness.

Specially Developed Hardware and Software

The projection area measures 12,360 × 7,850 pixels; each of the fifty-six projectors works independently in a decentralized network in order to maximize flexibility. Infrared emitters ensure that the infrared cameras can record images of the visitors in the nearly completely dark hall. Over several iterations, an adaptive software was developed that can record the people, follow their movements, and finally project their 'digital shadows.'

  • Opening


  • Location

    New York City

  • Client

    Park Avenue Armory

  • Partners

    Ai Weiwei StudioHerzog & de Meuron

  • Services

    Technical PlanningPrototypingSystem DevelopmentSoftware DevelopmentSystem IntegrationMechanics DevelopmentInteraction Design

Related Projects