Skip to main content

I tried iOS 16’s photo background remover and it blew me away

Apple rolled out a bunch of features with iOS 16. These include improvements in system apps like Messages, Maps, and Safari, a customizable lock screen, and more. But one of the most entertaining features is the ability to automatically remove the background from images — giving you a Photoshop-like edit with just a couple of taps.

It’s an intriguing feature that combines years of AI research and development. You can lift the subject from the background of an image by simply holding it and pasting it in Messages, Telegram, Safari, etc. I tried the feature on a wide range of photos. Some were captured by a DSLR, others with a smartphone camera with and without portrait mode. I also tested images of objects, pets, and many other pictures in challenging environments. After trying the feature on 10 different images, the results speak for themselves.

Related Videos

Objects

Sony Wf-1000XM4 earbuds with and without background
Prakhar Khanna/Digital Trends

The first image is of the Sony WF-1000XM4 earbuds. It was taken on a DSLR, which highlighted the earbuds and their case while blurring the packaging. I tapped and held on to the packaging and the system was smart enough to recognize the subject in focus. The AI precisely removed the background with clear edges where there was sufficient light but struggled with the shadows of the case and earbuds.

A person holding phone.
Prakhar Khanna/Digital Trends

I chose this second image because I captured it with the portrait mode effect on the iPhone — giving an artistic look to the picture. iOS 16 smartly understood the subject and picked up the phone with my hand. While I would have liked to have an option to pick just the phone instead of also saving my hand, it did a fine job with the edges. No complaints here.

Pets

A dog with tounge out with and without background.
Joe Maring/Digital Trends

Our Section Editor, Joe Maring, sent me this image of his dog in focus with the background far away. I tapped on the dog to lift him off the background, and it worked on the first try. It is worth noting that alongside the edges of the dog, the edges of his belt were discretely moved from the background. Overall, an impressive job.

Cat looking out of the window with and without bnackground
Prakhar Khanna/Digital Trends

While the AI did a good job in well-lit scenarios, Maring and I pushed the limits with this image of his cat sitting on the side of the window. The image is challenging for two reasons. First, it’s indoors, so there is not enough light in some areas around the cat. Secondly, the sunlight falls on the cat’s head, which could’ve confused iOS 16’s guesswork. But it didn’t. Apple’s new feature moved the cat from the background with precise edges even in low-lit areas like his tail.

Cat sitting on a table.
Prakhar Khanna/Digital Trends

Next up is another photo of a cat indoors, but this time with a person petting it. We were worried if iOS 16 would pick the objects in the background alongside the subject, but instead, it picked the human hand. It would have been much better if we could get an option to erase parts of the image we didn’t want – something we hope Apple is working on. The end result certainly isn’t bad, but it’s an example that there’s still room for improvement.

People

Prakhar Khanna using phone.
Prakhar Khanna/Digital Trends

I noticed that it was far easier for iOS 16 to remove the subject from a background when the image was captured in portrait mode. For instance, the above photo was taken in portrait mode on the iPhone 12. It did a decent job in maintaining the edges – even the hair. But, for some reason, iOS 16 created a blooming effect at my bottom. It could be because the portrait mode might have blurred that area, but it’s a strange anomaly nonetheless.

Prakhar Khanna with drink in hand
Prakhar Khanna/Digital Trends

Here, iOS 16 did a decent job in removing the face reflection of a person in the background. But if you are sitting with someone, and both of you are on the same axis, it’ll pick both of you. For instance, I cropped myself in this image, but the arm of the person sitting beside me couldn’t be removed even when I tapped on myself to lift the subject from the background. The above image wasn’t taken in portrait mode, but iOS 16 still did a decent job at the edges.

Prakhar Khanna sipping coffee
Prakhar Khanna/Digital Trends

This image was captured on the Huawei P50 Pro, which has one of the best portrait modes on a smartphone. But it got tricked in the area near my glasses and hair. iOS 16 went with the portrait mode data to cut out the edges very clearly but got deceived in the same areas where the camera didn’t do well. The edges around the hair could’ve been much better.

Prakhar with a laptop
Prakhar Khanna/Digital Trends

By contrast, this image was taken on a DSLR, and iOS 16 was smart enough to retain the laptop alongside me. It performed much better with the hair than in the previous image, cutting around the edges without picking up unwanted background.

Farhan Akhtar concert with and without background.
Prakhar Khanna/Digital Trends

One area where the iOS 16’s background remover struggles is when there is a crowd. The photo above is a great example. It was clicked on the iPhone 12 during a concert in a low-lit environment. While it did a decent job near the face, it couldn’t distinguish between the subject and the various hands in the crowd. Again, it would be nice of Apple to give us the option to erase unwanted objects to fix these little mistakes. But there’s only so much you can do with a tap and hold, thus making us miss 3D Touch even more.

Off to an incredible start

Overall, iOS 16’s background remover does a brilliant job in well-lit scenarios. It works well on pets and objects to clear out the background from the edges, but it seems like Apple is mostly relying on the portrait mode data to lift the subject from the background. As a result, when an image isn’t taken in portrait mode  — or something is blocking the subject — iOS 16 gets confused.

While these are valid complaints, it’s important to take all of this in context. This is the first available version of Apple’s background remover in the first Developer Beta of iOS 16. With iOS 16 not expected to officially launch until September or October, Apple has months ahead to tweak and refine the feature ahead of its public debut. Considering how good the feature already is at such an early stage, it’s exciting to think where it’ll be in a few months’ time.

Editors' Recommendations

Nothing Phone 2: news, release date and price rumors, and more
Nothing Phone 1 with Glyph lights active.

The Nothing Phone 1 made its debut in July 2022, and it had a reasonable amount of hype behind it due to the involvement of Carl Pei, a co-founder of OnePlus. It was a quirky phone due to the unique light show on the back that makes it stand out from the competition, but on the software front, it’s very similar to other Android phones out there. It received mixed reviews, though the consensus leaned more on the positive side.

This year, we’re expecting the Nothing Phone 2, as confirmed by Pei in January during MWC 2023. Here’s everything we know so far about the Nothing Phone 2!
Nothing Phone 2: design

Read more
6 years later, the iPhone X still does one thing better than the iPhone 14 Pro
iPhone X.

I’ve been an iPhone user since the very beginning, starting with the original iPhone. You know, the one with the 3.5-inch display that was perfect at the time, making it super easy to use a phone with one hand? As the years go by, the iPhone — and every other smartphone out there — just get bigger and bigger. We now have phones that with almost 7-inch displays, and honestly, I don’t understand how anyone can comfortably use these giant phones — especially if you have smaller hands!

With the iPhone, we’ve gone from 3.5-inch to 4-inches, then 4.7-inches to 5.8-inches, and now the standard 6.1-inch and 6.7-inch of the iPhone 14/iPhone 14 Pro and iPhone 14 Plus/iPhone 14 Pro Max, respectively. I personally use an iPhone 14 Pro as my primary device, and while I have gotten used to the 6.1-inch size over the past few years, I still think it’s too big. In fact, the last perfect size iPhone was the iPhone XS with the 5.8-inch display ... and I really wish Apple would bring it back.
5.8 inches was a perfect middle ground

Read more
How 3 Galaxy S23 Ultra camera features completely changed my vacation
Samsung Galaxy S23 Ultra back.

The Samsung Galaxy S23 Ultra has the most versatile set of cameras on any phone, and it's the phone I took with me to the Mobile World Congress (MWC) event in Barcelona, Spain, last month.

After my work at MWC 2023 was done, I had some free time to explore the beauty of Spain, and in doing so, the Galaxy S23 Ultra helped me capture some amazing shots that wouldn’t be possible on any other device. Here are the three Galaxy S23 Ultra camera features that I used extensively and which helped me make my trip memorable.
The clarity in 10x zoom is mind-blowing

Read more