‘Games art teams are ten years ahead of audio. We have some catching up to do – we need to ride their coat-tails...’
While some are celebrating the achievements of games sound designers, others are convinced that we are not making the best of the opportunities on offer. And with next-gen consoles and cloud computing in imminent prospect, the ‘others’ may well be right...
Mark Yeend is pulling no punches. His concerns that graphics are leaving audio behind sound melodramatic, but as Microsoft Games Studios Creative Director, he is particularly well placed to know.
Yeend describes both Microsoft’s Xbox One and the competing Sony PlayStation 4 as ‘incredible machines’. Set for provisional release in November, they promise to significantly advance the boundaries of gaming on all fronts. They share similar hardware specs (both use AMD ‘system on a chip’ processors that run the x86 instruction set), except in graphics processing power and their very different memory architectures.
Most easy to assess, the GPU (graphics processing unit – an AMD Radeon-based graphics engine) in the PS4 promises 50 per cent higher peak performance than the GPU in the Xbox One. Counting 1152 cores over the Xbox One’s 768, this is unlikely to be reflected in real-world use however. (There’s an interesting performance comparison on the ExtremeTech website conducted using PC models of both machines ahead of their launch.) Much harder to assess, memory and its use are much more significant.
Middleware – the tools used by developers to author games – has also seen dramatic advances over the past five years. For sound designers, this means that it has moved on from being code-based to more closely resembling music-recording equipment. ‘The ceiling has been raised,’ Yeend says. ‘The limitations that we used to meet have been lifted. Specifications should not define what we do.’
In Yeend’s world, this and next-gen consoles raise the curtain on a new era of audio design, where sound is treated as a series of systems rather than clips and soundtracks.
He describes this as ‘AI for audio’: By approaching sound as a rule set and an audio library, more complex content can be created and managed in a simpler and more time-effective way. Which brings us back to the use of memory in next-gen games consoles.
Beyond the sound barrier
The PlayStation 4 and Xbox One each pack 8GB of Heterogeneous Unified Memory Access (HUMA), which allows the CPU and GPU to share the same memory pool instead of having to copy and transfer data between them. Add in cloud computing, and we’re in a new world of opportunity…
Games have broken the linear paradigm of sound that was established by recorded music, radio and television drama, and cinema. The idea of locking events to time code that accompanies a definitive storyline has given way to an ‘adaptive cinematic experience’. But until now, games platforms have struggled to support this new paradigm.
Needing to provide many hours of gameplay with a smaller allocation of storage than a 100-minute movie, music composers and sound designers now look set to be liberated by the resources offered by next-gen games machines.
Next-gen platforms use fast processing and new processing structures with which to deliver audio and video. These allow immediate sounds and effects to be handled onboard, while music, ambience and other sounds that are more tolerant of latency can bring the immense power and resources of cloud processing farms into play. This allows the endless variation of natural sounds to be far more closely created using Yeend’s ‘AI for audio’ approach, and the resources of the cloud.
As in film, making the images for games is more glamorous than making the sound they accompany. Also as in film, these images falter and sometimes fail without sound. Sony’s prelaunch promotion claim that the PS4 is designed to deliver a ‘frictionless and seamless’ gaming experience demands these advances in audio. Without exceptional sound, its amazing graphics, its ability to play 4K/Ultra HD video and its ambitions to become a domestic ‘media hub’ are hamstrung.