The new future format

A pioneer within the field of media compression and technology – Professor David Taubman from UNSW’s School of Electrical Engineering and Telecommunications is shaping the world’s digital future. Through the creation of his globally adopted software and extensive contribution to international standards, Prof. Taubman’s theoretical and practical research has already impacted various fields, including geospatial, military and medical imaging, video broadcast and digital cinema. Prof. Taubman has contributed much of the core technology to a large family of media coding and communication standards known as JPEG 2000. Now, with the recent evolution of this technology, the impact of his work is only set to increase.

Professor Taubman explaining the comparison between HTJ2K and JPEG processing speeds (HTJ2K on top) from the HTJ2K white paper. Image credit: Shyani Seneviratna.

While Prof. Taubman has developed a number of major algorithms throughout his career, he’s best known for developing the core algorithm behind JPEG 2000 Part-1. JPEG 2000 tackles challenges not addressed by other compression standards, including images that run into terapixels in size, the compression of content which is highly accessible and the ability to interactively stream content from an image based upon a user’s interests. “It’s an extremely flexible family of standards but with one architecture,” explains Prof. Taubman. “The compressed representation itself is very flexible, so you can encode once and consume the content in many ways, at any time in the future.”

Following the creation of JPEG 2000 more than two decades ago, Prof. Taubman has since evolved his technology to create High-Throughput JPEG 2000 (HTJ2K) – a new set of coding tools within the J2K family that preserves all the features of the core technology originally invented by Prof. Taubman, but with extremely high speed in software, and an extremely small footprint in hardware.

This evolution makes JPEG 2000 the ideal choice for a huge range of applications that were previously computationally challenged. Applications include ultra-low latency compression, allowing for wire replacement in quality-sensitive video capture and dissemination networks, while adding the accessibility and interactivity features of JPEG 2000. By addressing new obstacles for a range of industries, HTJ2K is already making its mark.

“We’re seeing attention in the defence sector because they have huge content where speed is a problem,” explains Prof. Taubman.

“We’re also seeing interest in the media and entertainment sector, from professional content creation and archiving to point to point streaming of 4K video over traditional IP networks.”

Achieving diverse impact

Spanning an array of industries and sectors, the applications of JPEG 2000 and HTJ2K touch on a variety of industries – ranging from health to defence and beyond.

JPEG 2000 has been particularly vital for hostile imaging situations where users need to access imagery remotely. HTJ2K is further fuelling this presence.

“It has the potential to change the way imagery is collected and distributed on satellites and perhaps underwater drones and who knows what,” adds Prof. Taubman. “It’s an area where we see a lot of potential impact.”

For the health sector, the use of HTJ2K has the potential to change patient experiences and care outcomes. “It greatly speeds up the ability for health professionals to access any kind of medical imagery, including 3D imaging modalities such as CT and MRI scans,” explains Prof. Taubman. The technology speeds up the ability for specialists and doctors to access this content from anywhere in the world and interact with it. “Major players are building platforms on top of this right now, making it a significant area of growing impact,” adds Prof. Taubman. “It means a specialist in another country can easily get involved in a patient’s health management – they don’t have to be sitting in the hospital.”

For those within the area of cultural heritage preservation, JPEG 2000 has become the standard format used by the world’s leading institutions who have adopted the technology for use in archiving images, videos and documents. Examples include the Vatican, the Smithsonian Institution and a great many more. “It goes hand in hand with an open-source technology known as IIIF (International Image Interoperability Framework), which has been developed in that community for distributing huge archival imagery online,” adds Prof. Taubman. “Examples include providing individuals with access to their genealogical records or documents held behind glass.”

SMPTE summit in Hollywood, where the HTJ2K hardware video encoder was showcased by (L-R) Professor David Taubman, Graham Morton (Kakadu co-director) and Michael Smith (technical consultant to the major Hollywood studios).

Streaming into the future

While JPEG 2000 is one of the major archival and mezzanine formats for video and cinematography content distribution, HTJ2K could take this to the next level to become the basis for standard streaming connections.

The big obstacle to the adoption of the algorithm in streaming media (essentially wire replacement) however has been the market-need for hardware implementations. In the past, large technology companies often developed such implementations in-house. Now, many of these companies rely upon third parties to provide the core components behind the products they sell.

With this in mind, Prof. Taubman decided to commit resources to developing hardware. “When we demonstrated our hardware video encoder running for the first time, it got a lot of people excited,” says Prof. Taubman.

“They all wanted to see if it was possible, but no one had been prepared to try doing it themselves – what we have done has the potential to be quite transformative.”

Since then, Prof. Taubman and his team have been engaged in discussions with big chip manufacturers such AMD and Intel regarding the potential of HTJ2K. “It’s been a very rewarding experience because we’ve been working for about two years on trying to get hardware video encoding going with this new standard.”

As the impact of his work continues to evolve, Prof. Taubman compares the enduring outcomes of his research to that of a slow burning candle. “I’ve been lucky that I’ve been interested in things that are slightly less mainstream but are relevant to very difficult problems and those things don’t change as fast,” he says. “It’s made it possible for me to keep riding the wave.”

Hardware image credit: Shyani Seneviratna.