Immersive media: 360 video & 3D audio - Basics. Systems, techniques, standards

This article provides an overview of "immersive media" and the associated production and distribution systems. The term "immersive" is used in the case of digital media to describe the ability of a system to fully absorb a user in an audiovisual scene. "Immersive media" can be related to natural and/or computer-generated content. In this article [TE 5 690] and the following two [TE 5 692] and [TE 5 694] ), the focus is on 360° video and 3D audio. Immersion in a scene is strongly enhanced by interactivity, which can range from a simple finger swipe on a tablet to discover a 360° video scene, to rotations of the head of a user equipped with a visiocache to navigate the scene, accompanied or not by displacement.

The Cinéorama, patented in 1897 by Raoul Brimoin-Sanson, with a circular screen swept by 10 synchronized cinematographic projectors arranged in a circle, was the first 360° image projection process... well before the professional and then mass-market development of 360° video and video headsets at the end of the 20th and the beginning of the 21st century. For 3D audio, if we consider stereo as an ancestor, we can recall Clément Ader's retransmission of an opera over telephone lines in 1881, with the Théâtrophone... until the emergence of 3D audio effects in the 1990s, in PCs and game consoles. As for one of the earliest immersive and interactive systems, we might mention the Sensorama, patented in 1962 by Morton Heilig, which was a simulator of a motorcycle ride through New York, complete with screen, sound, wind and smell generators... right up to the more recent airplane piloting simulators or the immersive Birdly "bird flight" simulator (2016).

The development of immersive media has been boosted by the digital boom, particularly in digital video compression formats and Internet distribution networks, as well as by the spread of video games and portable devices (headsets, tablets and smartphones).

Applications for 360° immersive media include: event broadcasting (whether or not in parallel with a television stream), linear or interactive cinematography, educational applications, interpersonal and group communication (e.g. videoconferencing, or with online games, or in a virtual meeting place), guided tours, on-demand content viewing on visiocassette...

Beyond their interest in discovering a new technology,...