industry

Spatial Sound, Immersive Audio: What is it and is it here to stay?

Natalia Szczepanczyk, segment manager for Experiential and Immersive Applications at Holoplot, shares insights into the status quo of immersive entertainment and the advances in audio technology Holoplot brings

Tom Hanks’ The Moonwalkers utilises Holoplot’s software ecosystem to create immersive and spatial audio

First things first, what is Spatial Audio? Since the first storytellers mimicked the sound of a rampaging mammoth to illustrate the jeopardy in their hunt, or the Roman thunder machine roared in ancient theatres, sound effects have been used to enhance our understanding of and immersion in a story.

The stereo sound system was developed in the 1930s to mimic the way humans hear - with two ears. It was a method that was so successful that it’s lasted right up to the present day, with many of us still using stereo sound systems with entirely satisfying results. More recently, better microphones and line array technology have been utilised to enhance the story further, with better intelligibility across even bigger venues. But the ever bigger and advanced visual elements of live entertainment experiences make it more and more difficult to stick to traditional PA designs.

Today, there are no limits to the gigantic projected vistas that can be produced. In venues like Sphere in Las Vegas, images wrap around the entire audience, enveloping them in the story, or fully immersing them in a musical performance.

It’s clear that the vocabulary used to describe these new kinds of immersive and spatial experiences is running out of road. More refined definitions are required to do justice to the experience itself, not just the achievements and boundary-pushing innovations in the most immersive venues.

Describing an experience as immersive often doesn’t actually say much about the quality and magnitude of the audience’s immersion. I would argue, for instance, that some immersive audio solutions are more akin to surround sound with an advanced GUI.



According to neuroscientist Paul J Zak, a person’s attention during an experience plus emotional resonance with the content equals immersion. For me, this principle rings true and informs my definition below.

An immersive experience should be:

    • Natural: Sound needs to be localised and loudspeakers must not be identifiable as a source.
    • Intimate: Sound needs to not only move around me but also provide a perception of proximity. That’s ultimately what real life’s thrill is about.
    • Explorative: Sound needs to change in relation to my movement or the movement of the content.

In my opinion, an immersive experience should follow the definition by Paul Zak.

If those three points are fulfilled, I would argue that most people would say that they are “immersed” - but that conventional audio technologies are not equipped to meet those requirements.

Immersive doesn’t mean necessarily adding more loudspeakers into a venue or moving sound objects around a 360 surround. At Holoplot we try to think holistically and champion the interplay between different AV disciplines and technologies. From an audio perspective, creating an immersive experience starts with doing the basics better in a room, meaning cleaning things up to unprecedented levels to create a pristine base canvas you then add spatialisation effects to - applying the paint so to speak.

Utilising Holoplot’s technology allows you to use optimised audio beams, which provide coverage for any given space with pin-sharp accuracy. Its software algorithms deliver a consistent sound across the whole audience area with minimal loss of level over distance.

This has the profound effect that the ‘sweet spot’ is significantly enlarged and the audio isn’t perceived as coming from a specific loudspeaker position. From that point, Holoplot’s immersive toolkit provides creative opportunities others can’t reach, such as using reflections, creating Virtual Sources or bringing audio objects into the audience area.

Within a traditional stereo system, your sounds are limited to positions between two speakers; you can pan or move your audio between these two points, but no further. Also, with conventional systems, sound designers must often add more and more speakers to achieve coverage for the entire audience. This can equate to many hundreds of speakers, all doing different jobs. Each speaker needs power and signal. That’s a lot of cables, power and data. At Holoplot, a different approach offers a highly flexible, fully active product.



In 2011, Holoplot set out to throw out the rule book and redefine what it means to control sound. If light can be manipulated and controlled to the most precise degree, why not sound? The idea was simple: stop the sound going where you don’t want it to go and put it where you do want it. The execution was anything but simple and so began the journey into 3D Audio-Beamforming and exploration into Wave Field Synthesis (WFS) technology, combining the two in a unique product, the Matrix Array.

Delivering truly immersive experiences that can suspend disbelief requires all technical elements to gel together. Advanced sound control capabilities allow for the required flexibility to integrate all aspects of the experience to their fullest potential by working closely at point of conceptualisation with creatives to understand their aspiration and vision for the project.

With this kind of technology, previously thought impossible effects can now be realised, very much adding audio to an experienced designer’s bag of tricks. 3D Audio-Beamforming enables the precise, digital steering of sound by providing directivity control in both the vertical and horizontal axes. A single Matrix Array module can generate multiple sound fields simultaneously – each with its own content, equalisation, level, shape, and position with minimal spill between each field. This means you can place sound objects within a room without the usual limitations of the loudspeaker’s position or proximity to the audience member. Unlike conventional audio solutions, our audio objects can appear to be far away or, on the other hand, whisper in your ear. We showcase this feature at our demos by firing a whisper across the room with all participants lining up ‘on the beam’ to catch the words - step to the sides and you lose the audio.



Live experiences have a continuous need to up their game in order to compete with consumers’ digital alternatives. I mentioned earlier that describing an experience as immersive doesn’t actually say so much about the magnitude of the audience’s immersion and this could, in time, become a barrier to widespread immersive adoption. If audiences cannot consistently anticipate what they will experience it could slow consumer acceptance.

Lightroom in London is a prime example of a venue that truly delivers unique, artists-led content that immerses the audience visually and sonically. Their ambition was to turn a concrete basement with 11m high walls into a smooth-sounding gallery space without compromising 360-degree, full-height visuals on all four walls and the floor. Such a large rectangular concrete room, and a free-roaming audience comes with challenges that can only be addressed with the best, most innovative tech on the market. Auditoriums like Lightroom are very reverberant and easily excitable from an audio perspective.

This means that directional sound is hard to maintain as sound energy bounces off the walls, making it difficult to hear individual sounds, reducing intelligibility and making soundscapes tiring to listen to. During performances at Lightroom, images are projected onto every available surface, so the walls and floor need to be completely clear. There is no option to have speakers or projectors in view as this would spoil the immersive effect for the audiences.

Working in spaces like these can generate many challenges for a sound designer. When the venue opened last year, Gareth Fry, sound designer for the ​​opening show ‘DAVID HOCKNEY: BIGGER & CLOSER (not smaller and further away)’, and the team from 59 Productions, who conceived Lightroom, were looking for a suitable audio solution that would do justice to their unique storytelling approach. They quickly realised that conventional loudspeaker solutions would need multiple wall cavities or mounting positions on all four walls to achieve the desired immersive effects, compromising the visual ambitions of the project.

In this instance, Fry suggested Holoplot and a system design that was able to fit within the two predefined fixing positions within the room, yet delivered more creative tools that could be used to generate immersive effects. The system not only provided the answer for the current exhibit but also paved the way for future shows because it is easily reprogrammable.

Tom Hackley, sound designer for Lightroom’s latest show, The Moonwalkers, co-written and narrated by Tom Hanks, has benefitted from being able to cater for his new design, not by moving or adding to the audio hardware, but by utilising Holoplot’s software ecosystem.

It's not just the ease of adjusting the sound once your system is in place, the actual Matrix Array system is very different. With Holoplot, Hackley could precisely define the areas where he wanted sound and those where he didn’t.

The software allows sound designers to assign areas of the performance space that are to be ignored, so those reverberant flat surfaces or tricky balcony overhangs can be avoided with a simple programming change. 3D Audio-Beamforming uses intelligent software algorithms to create highly controlled, more efficient soundwaves than conventional speakers, ensuring that levels and quality remain consistent from point of origin to destination – even over large distances. This doesn’t just have application possibilities for performance; the techniques can be applied to many different situations.

In museums for example, where wearable tech is something of a standard, we see a greater push towards sound system utilisation to create zoned audio experiences. In a post pandemic world, sharing devices with large audiences is undesirable - thanks to innovations such as our own, audio can be targeted precisely at a defined area or around an exhibit, allowing the visitor to simply step in and out of the audio zone to experience the content.

The same applies for larger conference environments where multilingual content is reliant upon wearable technology and immersive elements are used to increase customer engagement.

At the recent reveal of the Polestar 4 at the Shanghai Car Show, an X1 system was deployed, making use of Holoplot’s audio control and multi-zoning abilities to create an inclusive, engaging and surprising launch event and exhibition display. The sound system not only allowed for the creation of dedicated listening zones with live, simultaneous language translation in English and Mandarin, but also delivered square metre pockets of sound around the car that allowed the public to discover details about the car’s new features, without zones interfering with each other. This unconventional distribution of information allowed the visitors to go on a journey and added an element of excitement to the display, increasing the time they stayed at the attraction.

One of the most desirable qualities of an immersive experience is an explorative element. Holoplot technology lets you create discoverable audio rather than blanket sound - experiences become tailored and changeable depending on how a visitor moves through an attraction. Sound plays an important role in guiding a visitor's discovery through such a space, directing the gaze to specific objects, underlining certain points in the visual content or setting the mood with background music.

A great example of this is the installation at Illuminarium Atlanta and Las Vegas. The venue lets visitors explore the 360-degree canvas on their own, engaging all their senses. The venues double up as educational spaces, bringing experiences to audiences they could never dream of in real life. Being able to teach children in such environments as opposed to through books helps to inspire and engage them, important factors when trying to raise the next generation of wildlife protectors or space scientists.

The opportunity to create customised audio zones for audience members also means that you no longer have to worry about spill between zones.

In retail, this principle could be applied by delivering specific information to tightly defined focus areas, such as a customer picking up a shoe in a retail venue and hearing more about the design concept, or someone waiting in a queue at a visitor attraction hearing specific content enhancing their enjoyment.



Holoplot’s ability to tightly control sound prevents polluting an already noisy environment with more content and allows designers to actively use focused audio as a tool to engage and excite audience members, whether they are experiencing that content in a shop, conference or entertainment venue. It’s an idea with endless possibilities and a creative tool that designers are starting to fully engage with.

I believe that immersive environments can improve a wide array of industries, but the execution has to be of high quality if they’re here to stay. Consumer adoption beyond the initial novelty hype will only occur if the technology is truly invisible and adds value to the experience. The ability to create truly spatial sound has endless possibilities and there are limitless ways to enjoy this new approach to sound design.