AI-enabled immersive technologies will drive more dynamic media futures

Published on the 20 July 2026

A dancer is captured inside the UNSW 3DXLab volumetric video arena, surrounded by more than 300 synchronised cameras.

A ballet performance being captured in the UNSW volumetric video arena. Photo: 3DXLab, UNSW.

Imagine a future where concert going means experiencing it on stage, where you watch the deciding moment in sport right there on the ground, where history comes to life in front of you.

New AI-enabled immersive technologies developed through UNSW’s 3DXLab are advancing dynamic media experiences just like these.

The multidisciplinary research lab, based at UNSW’s Paddington campus, has designed a world-leading stage to capture volumetric video.

Volumetric video differs from traditional filming in its ability to capture the 3D shape and depth of objects as they move in real time. Recorded objects or people can be placed in virtual environments where viewers can observe them in 360 degrees.

The stage, developed over more than two and a half years, combines a custom-designed camera rig with more than 300 synchronised cameras.

“This allows us to record people, performances and objects as dynamic, high-fidelity volumetric holograms, not flat video, as living 3D scenes you can move around, explore and interact with,” says Professor John McGhee, Director of the 3DXLab.

Visitors to the 3DXLab can walk around a 3D-captured performance, watching a dancer’s movements from all angles.

There is a growing international market for high-fidelity 4D content, including for entertainment and gaming, training, archival and health applications.

“The global tech race is on to expand 4D applications, from Taylor Swift experiences and sport simulations to archiving first-hand accounts of history, immersive training and health visualisations,” the award-winning 3D computer artist says.

“Next-generation immersive media – technologies that blend holograms, volumetric video capture and real-time spatial computing – creates innovative and ultra-realistic worlds.

“Promoting high-fidelity content will revolutionise entertainment and drive real-world impact across diverse fields.”

Cutting-edge rendering technology creates ultra-realistic detail

Creative industries face substantial financial and technical hurdles in producing realistic time-based volumetric video content for Extended Reality (XR) and Visual Effects (VFX) applications, Prof. McGhee says.

“Existing 4D video capture and processing tools are often slow, visually limited and lacking in scalable solutions for content sharing and collaboration.”

However, the 3DXLab stage uses Gaussian splatting, a cutting-edge rendering technique that rebuilds the captured object from millions of blended ellipsoidal shapes as it moves through space and time.

“Gaussian splatting renders much more realistic surfaces and intricate detail, such as violin strings or flyaway hairs, for an improved user experience,” he says.

The stage’s integrated processing pipeline uses AI to synthesise mass capture data to create a 3D rendering of the action.

A performance being captured in the UNSW volumetric video arena. Photo: 3DXLab, UNSW.

Bringing next-gen immersive tech to market

The 3DXLab’s work is being commercialised through spinout company Radiant Jelly, offering next-generation volumetric capture and holographic media production technologies.

The creative tech startup generates revenue through three channels: offering full-service production, licensing its software and hardware, and providing platform subscriptions. It has strong interest from national VFX network around AI/volumetric video services.

“Radiant Jelly’s high-quality, cost-efficient pipeline positions us to capture market share in the current SUS$3.58 billion global volumetric video industry, driven by content-creation revenue.”

Radiant Jelly is developing a cost-effective camera for use in a multi-camera volumetric rig and accompanying software that leverages AI. Its proprietary AI tech streamlines complex multi-camera and manual workflows to produce realistic time-based 3D video.

“The depth of the market for AI-enabled volumetric video is yet to be tested,” Prof. McGhee says. “With the development of XR [extended reality] glasses by major tech companies in collaboration with eyewear brands, new markets for volumetric video content are emerging all the time.

“It is already being used to capture live events as well as in education and training simulations, including for emergency services.”

In the not-too-distant future, it could be used to capture and revisit personal memories of our loved ones. “The authenticity of capturing someone as they actually are is really important to humans,” Prof. McGhee says.

“If we capture a memory of our grandparents, we want it to resonate with our embodied experience in all its fine detail.”