DxOMark adds bokeh and zoom to new smartphone camera scoring scale

DxOMark's testing protocol for scoring smartphone cameras adds zoom, bokeh

DxOMark Bokeh testing
If you want to know how good a smartphone camera is, DxOMark conducts the most in-depth reviews around. But after five years of evaluating smartphone cameras, the photography lab is introducing a major update to its test suite, adding two new categories — zoom and bokeh — and ramping up the importance of low-light and motion performance.

Since 2012, DxOMark has tested every phone with the same protocol, covering categories like exposure and contrast, color, autofocus, texture, noise, artifacts, and flash. The results are explained in great detail and then the scores from each category are brought together and a proprietary algorithm reduces them down to a single number out of 100. The top spot in the mobile charts, currently held by the HTC U11 with a score of 90, is highly coveted.

“Every single mobile that has been tested between 2012 and now, you can compare, because it followed the same protocol,” Clement Viard, senior director of image quality evaluation at DxOMark told Digital Trends. “But technology has changed so rapidly in the last five years that we have had to change the protocol.”

The impact of this change can be summed up neatly by looking at the iPhone 7 and 7 Plus. Because the old protocol didn’t consider the dual camera that Apple put in the 7 Plus to achieve 2x optical zoom and artificial bokeh, via Portrait mode, the two devices effectively had the same score. Retested with the updated suite, the iPhone 7 Plus achieves a significantly higher score of 88, compared to 85 for the iPhone 7.

DxOMark believes this change to its testing was necessary because of the growing demand from the public for high-quality zoom and bokeh effects, and improvements in the technology that are making it possible. The combination of powerful processors, dual-camera systems, improved sensors, and clever software tricks is taking smartphone photography to a whole new level.

“From the beginning of mobile photography, people at Nokia, Samsung, and other manufacturers were all asking the same thing,” Etienne Knauer, DxOMark’s senior vice president for sales and marketing, business solutions, explained to Digital Trends. “We want to be able to put in optical zoom, help us, and we want to have blurred background for portrait, help us. After all these years, now it’s becoming true.”

In addition to the new zoom and bokeh sub-scores, DxOMark has placed greater emphasis on low-light performance, testing levels all the way down to just 1 Lux. The Parisian photography lab has also added a motion element into its test suite to more accurately approximate real-world conditions, where both the photographer and subject are often moving.

For each smartphone tested under the new protocol, DxOMark will shoot 1,500 photos and more than two hours of video. The result is a 100-page report that serves as the foundation for the reviews and scores you see on the DxOMark website. The old scores will remain in archived form, but every phone tested by DxOMark from today will be subject to the new testing protocol.

Only a small sample of smartphones tested on the old scale have been retested with the new protocol. DxOMark chose the HTC U11 (90), Google Pixel (90), Apple iPhone 7 (85) and 7 Plus (88), Samsung Galaxy S6 Edge (82), iPhone 6 (73), and Nokia 808 (61).

“The ones that we’ve tested that were the best phones according to the old ranking are still the best devices on the new ranking,” Viard explained. “The Google Pixel and HTC U11 were really good before, and they’re still really good. For the iPhone 7 Plus, adding bokeh and zoom has given a big jump in image quality, which we believe reflects the user experience. Older devices like the iPhone 6 and Nokia 808 are dropping significantly, because they don’t have zoom or bokeh, and the autofocus was much slower back then.”

DxOMark will be announcing a number of new scores in the coming days and weeks, but for now, HTC’s U11 remains joint top of the chart on 90 with the Google Pixel, which has climbed one point. With the new iPhone and Google’s next Pixel just around the corner, it will be interesting to see if the photography lab crowns a new device as the best mobile camera.


Want more power, but faster? This new charging tech claims it can deliver

Chunky power bricks and slow charging could be a thing of the past with GaNFast technology from Navitas Semiconductors. By using an alternative to silicon, GaNFast reduces power consumption and boosts output.

Verizon’s deal could get you a free iPhone XR — but there’s some fine print

Verizon launched a new deal for its smartphones aimed at encouraging customers to open a new line. If you're willing and you want two new phones, you could get a free Samsung Galaxy S9, iPhone XR, or Pixel 3.

Android vs. iOS: Which smartphone platform is the best?

If you’re trying to choose a new phone and you’re not sure about the merits and pitfalls of the leading smartphone operating systems, then come on in for a detailed breakdown as we pit Android vs. iOS in various categories.

On a budget? We found the best affordable smartphones you can buy

Here are the best cheap phones for anyone working with a tight budget, whether you're a fan of stock Android or marathon battery life. Find out what you can get for under $500 or far, far less as we round up the best budget smartphones.

We tried all the latest and greatest smartphones to find the best of 2019

Smartphones are perhaps the most important and personal piece of tech on the planet. That’s why it’s important to pick the best phone for your individual needs. Here are the best smartphones you can buy.

Swatch and Mastercard team up for on-the-wrist, on-the-go mobile payments

Swatch has announced its Swatchpay technology is now available in Switzerland, enabling mobile payments from your Swatch watch. It works in a similar way to Apple Pay and Google Pay. Here's everything about it.

Do these Geekbench results accurately represent the Moto G7?

The Moto G6 range is still relatively new to the market, but rumors have already started about the Moto G7, which is expected some time in 2019. Apparently, a G7 Power version will be joining the G7, G7 Play, and G7 Plus.

Sony is showing something off at MWC -- will it be the Xperia XZ4?

Sony may have released the Xperia XZ3 in the past few months, but already it's preparing to release a follow-up, the Xperia XZ4. We're learning plenty about the phone now some details have started to leak out, and it's getting exciting.
Home Theater

Set your ears free with the best completely wireless earbuds

If you can't stand the tangle of cords, or you're just excited about completely wireless earbuds, you're going to need some help separating the wheat from the chaff. Our list serves up the best true wireless earbuds around.

Is this the first image of a Galaxy S10 being used in real life?

It won't be long now; With 2019 underway, the Samsung Galaxy S10 is almost here. Before it arrives, here's absolutely everything you need to know about all three of Samsung's next flagships.

Omron HeartGuide brings blood pressure monitoring to your wrist

High blood pressure leads to heart attacks, strokes, and many other health problems, so it's important to keep an eye on. Omron's HeartGuide is a fitness tracking watch that can also monitor your blood pressure from your wrist.

Google is buying mysterious smartwatch tech from The Fossil Group for $40 million

Google is about to step up its smartwatch game. The company has agreed to buy an unnamed smartwatch technology from The Fossil Group for a hefty $40 million. Considering the acquisition, it's clear Google is serious about smartwatches.

Learn how to play YouTube in the background on iOS and Android

We show you how to play YouTube in the background with apps such as Opera, Chrome, and Firefox -- along with the premium offerings like YouTube Premium -- whether you have an Android or iOS device.

From DIY to AAA, here's how to take a passport photo in 6 different ways

If you're applying for a passport or renewing one, you need to submit a photo in your official application. There are strict guidelines, but fortunately, it's something you can do at home. Here's how to take a passport photo.