Skip to main content
  1. Home
  2. Audio / Video
  3. News

Apple will use the iPhone’s camera to personalize spatial audio

Simon Cohen
By
Promotional logo for WWDC 2023.
This story is part of our complete Apple WWDC coverage

If you had blinked, you might have missed it: During the WWDC 2022 keynote address, Craig Federighi, Apple’s senior vice president of software engineering, focused on the new features we can expect in iOS 16. In his presentation, he briefly mentioned that you’ll be able to use an iPhone’s true depth camera to personalize spatial audio.

What does that mean? Well one of the keys to providing an immersive, 3D-like effect from spatial audio content using headphones is something called Head-Related Transfer Function (HRTF). It’s the science of understanding how everyone’s head shape and ear position can affect how you hear and process sounds in three dimensions.

Recommended Videos

It’s possible to deliver effective spatial audio experiences using a generic HRTF profile that works for most people, but the more your specific HRTF can be used, the better — in theory — spatial audio can be. Apple isn’t the first company to leverage a phone’s cameras in this way: Sony’s Headphones app, which it uses with products like the WH-1000XM4, XM5, and WF-1000XM4, will encourage you to take photos of both ears so that it can optimize the sound of its own spatial audio format, Sony 360 Reality Audio (360RA).

Related

Apple currently supports two types of spatial audio. Standard spatial audio lets you enjoy things like Dolby Atmos Music on any set of stereo headphones from Apple Music and other streaming services that support Atmos Music. Head-tracking spatial audio takes this concept to the next level by adding an extra level of immersion. When listening to spatial audio music formats, it can anchor certain elements of a song in 3D space, like the vocals. Moving your head makes it seems as though the source of vocals is coming from one direction instead of from all around you.

For movies, head-tracking spatial audio can make it seem as though you’re sitting in a full 5.1 home theater, including the sensation that your screen (like an iPhone, iPad, etc.) is the source of dialog and other key soundtrack elements.

We don’t yet know how Apple’s use of the iPhone’s true depth cameras will be used to create a more personalized spatial audio experience, but we’ll let you know as soon as we find out.

Editors' Recommendations

Topics
Simon Cohen
Simon Cohen
Contributing Editor, A/V
Simon Cohen covers a variety of consumer technologies, but has a special interest in audio and video products, like spatial…
Apple Music Sing brings karaoke singalongs to iPhones, Apple TV
Apple Music Sing.

Apple wants you to sing like everyone's listening with its new Apple Music Sing feature, a karaoke mode that works with the music streaming service's popular lyrics experience.

Available later this month on iPhone (11 and later), iPads (third-gen and later), and the 2022 model of Apple TV 4K (not earlier models, for some reason) to really max out the karaoke experience in living rooms, Apple Music Sing will be incorporated right into the Apple Music app, giving you the option to use a slider to control the volume of the vocals in Apple Music's song library so you can sing along. With the slider, you can tweak the volume to either blend your voice with the singer's or drop them out completely to go solo. Background vocals are also viewable independently from the main vocals for budding backup singers to follow more easily, and a Duet view separates multiple singers' lyrics on opposite sides of the screen.

Read more
FaceTime with spatial audio is nothing short of magic
A woman wears AirPods Pro while looking at her iPhone on public transportation.

The use of “magic” to describe tech is overblown, especially when it comes to Apple’s products. But hear me out on this one.

As someone who spends some of every day playing with new tech products, I’m frequently impressed by what I see. However, moments of genuine delight -- those times when I suddenly find myself smiling at something that just happened -- are rare. Still, I had one of those moments this weekend, and I want you to experience it, too.

Read more
A new Mercedes might be the best way to experience Apple Music’s spatial audio
Interior of a Mercedes-Benz with an Apple Music Spatial Audio sound system.

Mercedes has announced that it's the first car maker to give drivers a way to hear Apple Music's spatial audio content as it was meant to be experienced. Starting with the Mercedes-Maybach S-Class, the Mercedes-Benz S-Class, and the EQE, EQE SUV, EQS, and EQS SUV, spatial audio tracks in Dolby Atmos Music from Apple Music can be played without the use of an iPhone when buyers select the Burmester 4D or 3D sound system option for their vehicles.

Dolby Atmos support in cars isn't new. The Lucid Air became the first car to support the 3D audio format in 2021, and Mercedes itself added support for it on its Burmester systems that same year. But having a sound system that is Dolby Atmos capable, and being able to actually stream Dolby Atmos content to that system, are different things. Prior to Mercedes' announcement, getting a true Dolby Atmos Music experience from Apple Music meant using an iPhone with headphones or earbuds, or using an Apple TV connected to a Dolby Atmos-capable TV, soundbar, or home theater system.

Read more