Apple will use the iPhone's camera to improve spatial audio

Apple will use the iPhone’s camera to personalize spatial audio

If you had blinked, you might have missed it: During the WWDC 2022 keynote address, Craig Federighi, Apple’s senior vice president of software engineering, focused on the new features we can expect in iOS 16. In his presentation, he briefly mentioned that you’ll be able to use an iPhone’s true depth camera to personalize spatial audio.

What does that mean? Well one of the keys to providing an immersive, 3D-like effect from spatial audio content using headphones is something called Head-Related Transfer Function (HRTF). It’s the science of understanding how everyone’s head shape and ear position can affect how you hear and process sounds in three dimensions.

It’s possible to deliver effective spatial audio experiences using a generic HRTF profile that works for most people, but the more your specific HRTF can be used, the better — in theory — spatial audio can be. Apple isn’t the first company to leverage a phone’s cameras in this way: Sony’s Headphones app, which it uses with products like the WH-1000XM4, XM5, and WF-1000XM4, will encourage you to take photos of both ears so that it can optimize the sound of its own spatial audio format, Sony 360 Reality Audio (360RA).

Apple currently supports two types of spatial audio. Standard spatial audio lets you enjoy things like Dolby Atmos Music on any set of stereo headphones from Apple Music and other streaming services that support Atmos Music. Head-tracking spatial audio takes this concept to the next level by adding an extra level of immersion. When listening to spatial audio music formats, it can anchor certain elements of a song in 3D space, like the vocals. Moving your head makes it seems as though the source of vocals is coming from one direction instead of from all around you.

For movies, head-tracking spatial audio can make it seem as though you’re sitting in a full 5.1 home theater, including the sensation that your screen (like an iPhone, iPad, etc.) is the source of dialog and other key soundtrack elements.

We don’t yet know how Apple’s use of the iPhone’s true depth cameras will be used to create a more personalized spatial audio experience, but we’ll let you know as soon as we find out.

Editors' Recommendations