Skip to main content

What is semantic rendering, and how it improves your iPhone 11’s camera

iPhone 11 Pro Max rear triple camera
Julian Chokkattu/Digital Trends

The biggest improvements to Apple’s new iPhones are in the cameras, and not just because of the new ultra-wide-angle lenses on the iPhone 11, iPhone 11 Pro, and iPhone 11 Pro Max. The software powering the cameras is responsible for a significant leap forward in image quality thanks to improvements in computational photography techniques. One of the most interesting is semantic rendering, an intelligent approach to automatically adjusting highlights, shadows, and sharpness in specific areas of a photo.

What is semantic rendering?

In artificial intelligence, “semantics” refers to a machine’s ability to smartly segment information similar to how a human would. Different branches of machine learning may have different uses for semantic segmentation, but for photography, it starts with subject recognition.

In Apple’s case, the camera is specifically looking for any people within the frame, but it goes a level deeper than that. When the iPhone detects a human subject, Apple told Digital Trends it further differentiates between skin, hair, and even eyebrows. It can then render these segments differently to achieve the best results, creating a portrait that is properly exposed over the background.

[/pullquote]Just take a picture, and the phone will do the work in fractions of a second. [/pullquote]

To understand why this is so important, it helps to also understand how a standard camera works. Whether an older iPhone or a professional DSLR, a camera usually doesn’t know what it’s shooting. It knows the color and brightness of any given pixel, but it can’t glean any meaning about what’s actually in the frame. When you select the “portrait” color profile on a Nikon or Canon, for example, the camera is merely applying settings to specific color ranges of pixels commonly found in human subjects; it doesn’t really know if a person is present or not.

Such an effect is called a global adjustment, meaning it is applied to the entire photo equally. This is also how standard high dynamic range, or HDR, photos work: Highlights are lowered, shadows are raised, and midrange contrast might be enhanced — but without regard to what’s in the picture. This approach works well for subjects like landscapes, but it doesn’t always work for portraits.

iPhone 11 portrait mode Julian Chokkattu/Digital Trends

With semantic rendering, an iPhone 11 can apply local, rather than global, adjustments. This means a bright sky can have its brightness reduced to maintain color and detail, while the highlights on a person’s face won’t be reduced as much, preserving depth in the subject. Sharpness can also be applied to the skin and hair in different strengths.

Photographers have been doing this kind of retouching by hand in programs like Adobe Photoshop for years, but the enhancements are applied instantly on an iPhone 11.

How do you use it? Just take a picture, and the phone will do the work in fractions of a second. Know that semantic rendering only affects human portraits; other types of photos receive the standard HDR treatment. It is not limited to portrait mode — any photo with a human subject is automatically a candidate for semantic rendering.

Computational photography — which incorporates everything from HDR to depth-sensing portrait modes — enables phone cameras to surpass the physical limitations of their small lenses and sensors. Apple’s semantic rendering is among the next evolution of these technologies — Google has been using similar machine learning to power the camera in its Pixel smartphones.

While the tech powering it is complex, its goal is simple. By giving the iPhone the ability to know when it’s looking at a person, it sees the world a little more like we do, leading to pictures that look natural and more true to life.

Editors' Recommendations

Daven Mathies
Former Digital Trends Contributor
Daven is a contributing writer to the photography section. He has been with Digital Trends since 2016 and has been writing…
iPhone 16: news, rumored price, release date, and more
A person holding the Apple iPhone 15 Plus and Apple iPhone 15 Pro Max.

Apple iPhone 15 Pro Max Andy Boxall / Digital Trends

Though Apple only launched the iPhone 15 and iPhone 15 Pro recently, rumors have already begun about the iPhone 16 lineup coming in 2024. It's hard to believe, but it's true!

Read more
This is one of the best iPhone 16 leaks I’ve seen yet
A green iPhone 15 in hand in front of plant.

Though the iPhone 15 series is still relatively new, the rumor mill for the iPhone 16 has already begun. And since we’re in the era of spatial computing with the Apple Vision Pro, some changes need to be made to future iPhones to make compatible content.

So far, we’ve seen a few different design prototypes for the iPhone 16, at least with the base model. And though I always spring for the Pro models instead of the non-Pro versions, the design of the base iPhone 16 models has me excited for what's to (possibly) come.
A call back to the iPhone X

Read more
I’m in love with these Lunar New Year iPhone accessories
OtterBox Lumen Series Year of the Dragon for iPhone 15 Pro with mophie Powerstation Plus 10K CNY Dragon.

Lunar New Year is on Saturday, February 10, 2024. This is the start of the new year based on the lunar calendar; this time, we are celebrating the Year of the Dragon, and 2024 is associated with the element of Wood. Essentially, this is the year of the Wood Dragon.

I’m a rabbit myself, but I’ve always been a fan of dragons — they're one of my favorite mythological creatures. According to the Chinese zodiac, those born in the year of the dragon are charismatic, intelligent, powerful, and naturally lucky or gifted. Definitely unique and one of a kind!

Read more