Skip to main content

Apple will use the iPhone’s camera to personalize spatial audio

If you had blinked, you might have missed it: During the WWDC 2022 keynote address, Craig Federighi, Apple’s senior vice president of software engineering, focused on the new features we can expect in iOS 16. In his presentation, he briefly mentioned that you’ll be able to use an iPhone’s true depth camera to personalize spatial audio.

What does that mean? Well one of the keys to providing an immersive, 3D-like effect from spatial audio content using headphones is something called Head-Related Transfer Function (HRTF). It’s the science of understanding how everyone’s head shape and ear position can affect how you hear and process sounds in three dimensions.

It’s possible to deliver effective spatial audio experiences using a generic HRTF profile that works for most people, but the more your specific HRTF can be used, the better — in theory — spatial audio can be. Apple isn’t the first company to leverage a phone’s cameras in this way: Sony’s Headphones app, which it uses with products like the WH-1000XM4, XM5, and WF-1000XM4, will encourage you to take photos of both ears so that it can optimize the sound of its own spatial audio format, Sony 360 Reality Audio (360RA).

Apple currently supports two types of spatial audio. Standard spatial audio lets you enjoy things like Dolby Atmos Music on any set of stereo headphones from Apple Music and other streaming services that support Atmos Music. Head-tracking spatial audio takes this concept to the next level by adding an extra level of immersion. When listening to spatial audio music formats, it can anchor certain elements of a song in 3D space, like the vocals. Moving your head makes it seems as though the source of vocals is coming from one direction instead of from all around you.

For movies, head-tracking spatial audio can make it seem as though you’re sitting in a full 5.1 home theater, including the sensation that your screen (like an iPhone, iPad, etc.) is the source of dialog and other key soundtrack elements.

We don’t yet know how Apple’s use of the iPhone’s true depth cameras will be used to create a more personalized spatial audio experience, but we’ll let you know as soon as we find out.

Editors' Recommendations

Simon Cohen
Contributing Editor, A/V
Simon Cohen covers a variety of consumer technologies, but has a special interest in audio and video products, like…
Netflix adds spatial audio to select movies and shows for all subscribers
Netflix app icon on Apple TV.

Netflix has announced that it has added spatial audio to a variety of movies and shows on its streaming video service. The audio format is available to all subscribers regardless of their subscription tier or their country. To find spatial audio-enabled content, type "spatial audio" into the Netflix search bar. Any of the titles shown in the results will support the feature, according to the company's blog post.

Spatial audio refers to any audio that goes beyond the traditional stereo or 5.1 surround sound experience. Until now, the only way you could hear spatial audio on Netflix content was via Dolby Atmos, which is a spatial audio format that requires content that has been recorded in Dolby Atmos. It must also be played on a device that specifically supports Dolby Atmos. On top of that, Netflix only makes Dolby Atmos versions of its movies and shows available to its highest subscription tiers, further limiting who can access it.

Read more
Apple Music relaunches One Mix DJ series in spatial audio
Apple Music One Mix DJ series in spatial audio.

Fans of DJ mixes now have a new reason to consider switching to Apple Music. The streaming music service has relaunched its One Mix collection of DJ mixes with the addition of Dolby Atmos Music-powered spatial audio tracks.

To kick off the new feature, Apple Music has partnered with Detroit techno pioneer, DJ, and producer Jeff Mills. His Outer to Inner Atmosphere: The Escape Velocity Mix is an Apple Music exclusive, and the first of many DJ mixes to get the Dolby Atmos treatment.

Read more
YouTube TV finally adds picture-in-picture for iPhone and iPad
YouTube TV shown picture-in-picture atop a web browser on an iPhone.

After months of promises and and extreme patience from customers, YouTube TV finally has added a picture-in-picture feature (also referred to as PiP) for iPhone and iPad. The company confirmed the rollout this week, and it started appearing in the app a few hours later, apparently triggered by a server-side update.

The feature allows you to continue watching a show in a smaller window while doing something else on your device.

Read more