Skip to main content

DxOMark adds bokeh and zoom to new smartphone camera scoring scale

DxOMark's testing protocol for scoring smartphone cameras adds zoom, bokeh

DxOMark Bokeh testing
If you want to know how good a smartphone camera is, DxOMark conducts the most in-depth reviews around. But after five years of evaluating smartphone cameras, the photography lab is introducing a major update to its test suite, adding two new categories — zoom and bokeh — and ramping up the importance of low-light and motion performance.

Since 2012, DxOMark has tested every phone with the same protocol, covering categories like exposure and contrast, color, autofocus, texture, noise, artifacts, and flash. The results are explained in great detail and then the scores from each category are brought together and a proprietary algorithm reduces them down to a single number out of 100. The top spot in the mobile charts, currently held by the HTC U11 with a score of 90, is highly coveted.

“Every single mobile that has been tested between 2012 and now, you can compare, because it followed the same protocol,” Clement Viard, senior director of image quality evaluation at DxOMark told Digital Trends. “But technology has changed so rapidly in the last five years that we have had to change the protocol.”

The impact of this change can be summed up neatly by looking at the iPhone 7 and 7 Plus. Because the old protocol didn’t consider the dual camera that Apple put in the 7 Plus to achieve 2x optical zoom and artificial bokeh, via Portrait mode, the two devices effectively had the same score. Retested with the updated suite, the iPhone 7 Plus achieves a significantly higher score of 88, compared to 85 for the iPhone 7.

DxOMark believes this change to its testing was necessary because of the growing demand from the public for high-quality zoom and bokeh effects, and improvements in the technology that are making it possible. The combination of powerful processors, dual-camera systems, improved sensors, and clever software tricks is taking smartphone photography to a whole new level.

“From the beginning of mobile photography, people at Nokia, Samsung, and other manufacturers were all asking the same thing,” Etienne Knauer, DxOMark’s senior vice president for sales and marketing, business solutions, explained to Digital Trends. “We want to be able to put in optical zoom, help us, and we want to have blurred background for portrait, help us. After all these years, now it’s becoming true.”

In addition to the new zoom and bokeh sub-scores, DxOMark has placed greater emphasis on low-light performance, testing levels all the way down to just 1 Lux. The Parisian photography lab has also added a motion element into its test suite to more accurately approximate real-world conditions, where both the photographer and subject are often moving.

For each smartphone tested under the new protocol, DxOMark will shoot 1,500 photos and more than two hours of video. The result is a 100-page report that serves as the foundation for the reviews and scores you see on the DxOMark website. The old scores will remain in archived form, but every phone tested by DxOMark from today will be subject to the new testing protocol.

Only a small sample of smartphones tested on the old scale have been retested with the new protocol. DxOMark chose the HTC U11 (90), Google Pixel (90), Apple iPhone 7 (85) and 7 Plus (88), Samsung Galaxy S6 Edge (82), iPhone 6 (73), and Nokia 808 (61).

“The ones that we’ve tested that were the best phones according to the old ranking are still the best devices on the new ranking,” Viard explained. “The Google Pixel and HTC U11 were really good before, and they’re still really good. For the iPhone 7 Plus, adding bokeh and zoom has given a big jump in image quality, which we believe reflects the user experience. Older devices like the iPhone 6 and Nokia 808 are dropping significantly, because they don’t have zoom or bokeh, and the autofocus was much slower back then.”

DxOMark will be announcing a number of new scores in the coming days and weeks, but for now, HTC’s U11 remains joint top of the chart on 90 with the Google Pixel, which has climbed one point. With the new iPhone and Google’s next Pixel just around the corner, it will be interesting to see if the photography lab crowns a new device as the best mobile camera.

Editors' Recommendations

Simon Hill
Former Digital Trends Contributor
Simon Hill is an experienced technology journalist and editor who loves all things tech. He is currently the Associate Mobile…
The best smartphone case brands in 2023: the 20 best ones
iPhone 12 Mini

Did you just spend three or even four figures on a new smartphone? You'll probably be wanting to keep it pristine and safe from damage, and the best way to ensure that is with a good protective case. However, not every case is created equal. While you can pick up a generic gel case from anywhere for a few bucks, if you want real protection, you're better off looking at some of the best case brands for a more expensive, but much more reliable smartphone case.

Phone cases aren't just about protection, though. The best phone cases offer protection, sure, but they also boast stylish designs, kickstands, or even backpacks or wallets to hold credit cards or cash. But where should you be looking for your new case? We've got the lowdown on the best phone case brands out there right now, so you can zero in on the right case for you.

Read more
I tried to replace my GoPro with this new phone and its clever camera
The Asus Zenfone 10, along with the Samsung Galaxy S23 Ultra, Apple iPhone 14 Pro, and the GoPro Hero 11 Black.

The Asus Zenfone 10's camera is attached to a gimbal, and with it comes the promise of shooting steady, shake-free video even when you’re moving around.

Does this mean it can take on the mighty GoPro, and perhaps mean you only need to carry around one device instead of two? To find out, we put the Zenfone 10 against the latest GoPro camera and two of its smartphone peers.
Understanding the Zenfone 10’s gimbal
The gimbal can be seen working in the viewfinder Andy Boxall/Digital Trends

Read more
You aren’t ready for this Galaxy S23 vs. iPhone 14 Pro camera test
Deep purple iPhone 14 Pro and Cream Galaxy S23 crossed over

Samsung’s Galaxy S23 is here, and it's quickly become one of the best phones you can buy in 2023. For $800, you’re getting a small but mighty phone with Snapdragon 8 Gen 2 for Galaxy chipset, long-lasting battery life, and a powerful triple lens camera system with a 50-megapixel main shooter.

But how does one of the best Android phones stack up against Apple’s smallest flagship, the iPhone 14 Pro? It has just as many cameras as the Galaxy S23, a powerful 48MP main camera, and costs $200 more than Samsung's handset.

Read more