DxOMark adds bokeh and zoom to new smartphone camera scoring scale

DxOMark's testing protocol for scoring smartphone cameras adds zoom, bokeh

DxOMark Bokeh testing
If you want to know how good a smartphone camera is, DxOMark conducts the most in-depth reviews around. But after five years of evaluating smartphone cameras, the photography lab is introducing a major update to its test suite, adding two new categories — zoom and bokeh — and ramping up the importance of low-light and motion performance.

Since 2012, DxOMark has tested every phone with the same protocol, covering categories like exposure and contrast, color, autofocus, texture, noise, artifacts, and flash. The results are explained in great detail and then the scores from each category are brought together and a proprietary algorithm reduces them down to a single number out of 100. The top spot in the mobile charts, currently held by the HTC U11 with a score of 90, is highly coveted.

“Every single mobile that has been tested between 2012 and now, you can compare, because it followed the same protocol,” Clement Viard, senior director of image quality evaluation at DxOMark told Digital Trends. “But technology has changed so rapidly in the last five years that we have had to change the protocol.”

The impact of this change can be summed up neatly by looking at the iPhone 7 and 7 Plus. Because the old protocol didn’t consider the dual camera that Apple put in the 7 Plus to achieve 2x optical zoom and artificial bokeh, via Portrait mode, the two devices effectively had the same score. Retested with the updated suite, the iPhone 7 Plus achieves a significantly higher score of 88, compared to 85 for the iPhone 7.

DxOMark believes this change to its testing was necessary because of the growing demand from the public for high-quality zoom and bokeh effects, and improvements in the technology that are making it possible. The combination of powerful processors, dual-camera systems, improved sensors, and clever software tricks is taking smartphone photography to a whole new level.

“From the beginning of mobile photography, people at Nokia, Samsung, and other manufacturers were all asking the same thing,” Etienne Knauer, DxOMark’s senior vice president for sales and marketing, business solutions, explained to Digital Trends. “We want to be able to put in optical zoom, help us, and we want to have blurred background for portrait, help us. After all these years, now it’s becoming true.”

In addition to the new zoom and bokeh sub-scores, DxOMark has placed greater emphasis on low-light performance, testing levels all the way down to just 1 Lux. The Parisian photography lab has also added a motion element into its test suite to more accurately approximate real-world conditions, where both the photographer and subject are often moving.

For each smartphone tested under the new protocol, DxOMark will shoot 1,500 photos and more than two hours of video. The result is a 100-page report that serves as the foundation for the reviews and scores you see on the DxOMark website. The old scores will remain in archived form, but every phone tested by DxOMark from today will be subject to the new testing protocol.

Only a small sample of smartphones tested on the old scale have been retested with the new protocol. DxOMark chose the HTC U11 (90), Google Pixel (90), Apple iPhone 7 (85) and 7 Plus (88), Samsung Galaxy S6 Edge (82), iPhone 6 (73), and Nokia 808 (61).

“The ones that we’ve tested that were the best phones according to the old ranking are still the best devices on the new ranking,” Viard explained. “The Google Pixel and HTC U11 were really good before, and they’re still really good. For the iPhone 7 Plus, adding bokeh and zoom has given a big jump in image quality, which we believe reflects the user experience. Older devices like the iPhone 6 and Nokia 808 are dropping significantly, because they don’t have zoom or bokeh, and the autofocus was much slower back then.”

DxOMark will be announcing a number of new scores in the coming days and weeks, but for now, HTC’s U11 remains joint top of the chart on 90 with the Google Pixel, which has climbed one point. With the new iPhone and Google’s next Pixel just around the corner, it will be interesting to see if the photography lab crowns a new device as the best mobile camera.


Get up close and personal with this telephoto lens for your phone

Moment is replacing its aging 60mm telephoto lens with a new 58mm tele lens, redesigned from the ground up for the latest iPhone, Pixel, and Galaxy phones. Mount it onto the phone via a case and get closer with 2x optical zoom.

Save up to $850 with the best smartphone deals for November 2018

Need a better phone but don't want to spend a fortune? It's never a bad time to score a new smartphone and save some cash. We rounded up the best smartphone deals available that can save you as much as $850.
Product Review

Google’s Pixel 3 is a hair away from pocket-sized perfection

Google’s Pixel 3 smartphone is the best Android phone you can buy. It doesn’t have the best looks or the best hardware, but you’ll be hard pressed to find better software and unique A.I. functionalities.
Product Review

Mediocre battery and a big notch slight Google's otherwise perfect Pixel phone

Google’s Pixel 3 XL has two big flaws: The gigantic notch on the front, and mediocre battery life. That being said, this is the best Android experience you can find in a smartphone today.

How does fast charging work? Here’s every single standard compared

Modern smartphones can charge in mere minutes instead of hours. How does fast charging work? Here's a guide to the most popular standards, including Qualcomm Quick Charge, Apple fast charging, OnePlus Dash Charge, and more.

The Best Black Friday Deals from Best Buy in 2018

We've been hard at work assembling all the best Black Friday deals Best Buy offers in 2018 and putting them in one place to save you time and money this holiday season. From laptops to TVs, game consoles to smart speakers and much more…

Samsung Galaxy Note 10 display could be bigger than the iPhone XS Max screen

The Samsung Galaxy Note 9 was only released a few months ago, but Samsung is already working on a follow-up. Not much is known about the Samsung Galaxy Note 10 just yet, but we do have a few details.

Keep your phone organized with one of the best file managers for Android

Your smartphone has a limited amount of storage space and all sorts of files tend to accumulate if you let them. To keep things in order and find what you need, you should snag one of the best file managers for Android.

Upcoming Honor View 20 may forgo the notch for new display technology

Nearly a year after Honor released the spectacular Honor View 10, the Chinese smartphone giant appears to be hard at work on its successor, the Honor View 20. Here's everything we know about the upcoming midrange powerhouse.

The best Walmart Black Friday deals in 2018

Walmart has historically been the undisputed king of Black Friday deals. The mega-store is known for offering deals on products in almost every category, from smart TVs to children’s toys. We're combing through every deal as it is…

The Google Photos iOS app gets a bit better, now has depth control

Google updated the iOS version of the Google Photos app to now take advantage of the depth data that can be captured by the iPhone's camera in Portrait Mode. The new feature is already available in the Android app.

T-Mobile-Sprint merger could close as early as first quarter of 2019, exec says

T-Mobile and Sprint are finally merging. After a few failed attempts, the two companies announced their merger at the start of 2018. Once it's complete, the new T-Mobile could be better positioned to take on the likes of Verizon and AT&T.

Apple confirms it’s collecting data on foot to improve its Maps app

Apple has been using cars and satellites to improve its Maps app, and it's now confirmed it's also using tech-laden pedestrians to collect data from the streets in a bid to provide better walking directions.

Black Friday 2018: The best deals so far

Black Friday is the biggest shopping holiday of the year, and it will be here before you know it. If you can't wait until November 23 to start formulating a shopping plan, we've got you covered.