UNSW Centre for Interactive Cinema Research

Conversations @the Studio
Edols & Elliot Glass Studio Brookvale. Spherical view

Conversations@the Studio presents an innovative mixed-reality narrative model of the Powerhouse Museum (Sydney)’ Decorative Arts collection within a uniquely designed Intelligent Interactive Visualisation Environment. It takes the natural navigation of a real-world situation – a contemporary Glass Studio – as its point of departure, using this as a framework for organising a set of narrative formations that further elaborate the thematics of the collection. By visualising a 360 degree global video recording, made on location at the Glass Studio, it provides the tele-present experience of an actual visit to the Studio, giving full interactive freedom to the viewer’s gaze.
Installation at the Powerhouse Museum

This project aims to dramatically enhance the presentational flexibility of museum information delivery through the application of an immersive system of cinematic visualisation, modelling the integration of three kinds of interactive narrative. By allocating selective agency to both objects displayed and to viewers within a virtual environment, the system allows museum visitors to invest even static artefacts with a range of vivid narrative purpose. This project has been adjusted to accommodate the fact that only 50% of the requested ARC budget was awarded. This has necessitated a concentration on the experimental demonstrator for the Powerhouse Museum and web site evaluating and documenting the study. It has also necessitated a simplification of the original model whereby the presence of narrative agents within the IIIE has been replaced by an interaction strategy where the multilayered narrative extensions are embedded within the global video recording. This is articulated by multiple audio streams, and by conjunctions between the global scene and close-up sequences that detail the glass handling process.
Space-Time relation of close ups in the 360º global recording

Conversations@the Studio sets a new museological multi-media benchmark. Its interactive and immersive frameworks allow the viewer to experience themselves present in a distant space where artists are engaged in their daily creative craft making activities. They can move their gaze freely anywhere in the live performance space whose unedited real time narration converges with the real time engagement of the viewer. This space-time continuum between the local (Powerhouse Museum) and remote (glass studio) location enables the viewer to achieve an immediacy of experience that has hitherto not been achievable in multi-media exhibits. This is reinforced by the dynamic surround sound reproduction of the glass studio environment that is sensitive to the virtual position of the viewer’s gaze. A key feature of Conversations@the studio is the dynamic tracking of the viewer’s gaze in both space and time that visually magnifies specific events of interest. This polychronic cinematic operability gives the viewer a multi-visual faculty that moves seamlessly between an apprehension of the total active space and the amplification of its significant details. The movement is further emphasised by the viewer’s freedom to move between the live ambient surround sound recording made in that space, and the focussed commentaries of the glass studio artists which refer to specific processes and events that are unfolding.

Project Development

The first stage of research in 2004 involved: The analysis of the core components to be integrated from the experimental models; the planning and design of the IIIE; the 360 degree global recording; testing with sample audio and video; design of the interface system; and post-production of the 360 degree video and audio recording. The second stage of research involved the completion of all the implementation tasks and the installation of the demonstrator, in 2005, at the Powerhouse Museum as part of the museum’s new permanent decorative arts wing for an exhibition entitled Inspired! design across time. The final stage of the research ensued from the evaluation of public user interaction with the installation at the Powerhouse Museum, and adjustments of the interaction design to facilitate improved functionality.
Edols & Elliot Glass Studio Brookvale. Equirectangular view

The highly successful installation at the Powerhouse takes the form of a custom designed three-meter diameter hemispherical fibreglass projection screen that is mounted vertically so that viewers look straight ahead into its immersive concave projection space. A high-resolution projector fitted with a unique 360 degree fisheye lens is mounted in front of the screen, and the projection is augmented by a 5 channel surround sound system. The specifically developed user interface is an over-sized illuminated track ball that allows navigation of the global video recording, and which changes colour in dynamic relation to the content displayed on the screen. This latter functionality is adapted to signalling the availability of close-up sequences within the global scene. Additional controls activate the voice-over commentary and offer a fast forward function. The IIIE forms the central digital focus of the Powerhouse Museum’s new permanent gallery showcasing artefacts from its Decorative Arts and Design collection. The IIIE utilizes an integrated design strategy that is driven by the specific nature of the Museum’s decorative arts content, enriching the content by offering multiple layers of multimedia production that augment methods of documentation using 360 degree global recording. Innovative visualisation and interface strategies incorporate these recordings in an environment where the viewer can encounter and explore the content in a dynamic manner.
Conversations@the Studio installation model

Technology

  • Conversations@the Studio utilises a 3m dome as the surface for 180 degree projection made possible by a projector equipped with a fish eye lens. The lens is close to the centre of the sphere to ensure total coverage of the screen with minimal distortion. The size and shape of the projection set-up covers the peripheral vision of the user standing directly in front of it and thus ensures a truly immersive experience.
    Installation 3D model

  • The video for Conversations@the Studio was shot with a Ladybug camera system from Point Grey Research. It allows digital spherical recordings at 360 degree horizontal and 240 degree vertical field of view. The camera has a tightly packed cluster of 6 CCD sensors with wide angle lenses and a slight overlap between the images. In post production the individual frames were colour and geometrically corrected and stitched to a high resolution equirectangular image (3600×1800 pixel). Custom algorithms where used to double the frame rate from 15 (the camera’s maximal frame rate) to 30 frames per second. The final delivery format of this 30 min movie is MPEG-2 at a very high bit-rate to allow for high quality playback.

  • The sound was recorded using a 4 channel Sound-Field microphone accompanied by a 5.1 Holophone. The individual directional sound channels are mapped to a point of view within the video and synchronised for playback in post production.

  • A custom built 3D software engine takes the high resolution video stream from a disk array and uses it as a texture map projected on the inside of a sphere. The geometric correction for the fisheye lens is undertaken through a high resolution distortion mesh in the render pipeline.

    The video texture is translated inside the sphere according to the point of view of the user, this simulates “looking around” with in the scene.

    A look up table supplies the playback engine with location and time/duration of a video close-ups or hot-spots. If the point of view is within a hot-spot area in time, a close up is superimposed as a second layer on top of the main movie.
    Distrorted video texture
  • Integrated projector stand

    The spherical movie is projected inside the dome with a coverage of 180 degree. A track ball integrated in the projector stand allows the user to rotate the projection freely while the movie is still running. Together with the image the multi-channel sound field is rotated accordingly. A simple vector based panning algorithm is used to distribute the multi-channel sound direction dependent on the user’s principle point of view.
    A control button activates a voice over commentary channel, whose content is related to the point of view and the time within the main spherical movie. This voice channel is mixed into the centre channel of the directional ambient audio. A second control button allows fast forward play back of the movie.

ARC Investigators: Dennis Del Favero, Jeffrey Shaw, Neil Brown, Volker Kuchelmeister, Nikos Papastergiadis, Scott McQuire, Andy Arthurs, Sarah Kenderdine, Kevin Sumption, Grace Cochrane
Programmer: Joachim Tesch
Project Funding: ARC LP0453638, Australia Council for the Arts, Powerhouse Museum
2005-2012
Industry Partner: Museum of Applied Arts & Sciences (MAAS), Sydney
  • Australasian Planetarium Society Conference, Horizon Planetarium, Perth, 2008
  • Darling River Journey, Back of Bourke Exhibition Centre, Bourke, 2005-2007
  • Inspired! Design across time, Powerhouse Museum, Sydney, 2005-2012

Directors: Jeffrey Shaw, Andy Arthurs, Neil Brown, Grace Cochrane, Dennis Del Favero, Volker Kuchelmeister, Scott McQuire, Nikos Papastergiadis, Sarah Kenderdine, Kevin Sumption

Glass Artists: Alexandra Chambers, Benjamin Edols, Kathy Elliott, Tom Rowney

Production and Technical Co-Ordinator: Volker Kuchelmeister

Lead Software Engineer: Joachim Tesch

Production Manager: Damian Leonard

Project Manager: Kate Dennis

Human Interface Designers: Volker Kuchelmeister, Jeffrey Shaw

Camera Operator: Greg Ferris

Lighting Designer: Max Harrison

Production Assistant: Sue Midgley

Spherical Video, Close-ups and Voice Commentaries: Volker Kuchelmeister

Sound Field Design: Andy Arthurs

This project has been funded by the Commonwealth Government through the Australian Research Council’s research funding body and the Australia Council’s arts funding and advisory body.

The project has been co-produced by the UNSW iCinema Centre for Interactive Cinema Research, The University of Melbourne, Queensland University of Technology and the Powerhouse Museum, Sydney.

The producers wish to thank the following for their generous support in the realization of the project:

Paul Bourke

Edols & Elliott Glass Studio

NoseCone Australia P.L

Videocraft Equipment Pty Ltd

Xenon Systems Pty Ltd

Sound on Stage Pty Ltd

Brendan Lloyd

Powerhouse Museum Staff

Integral Event Management

Funded by:
Australian Research Council Linkage Project: Reformulating museological narrative using three models of cinematic interactivity.