Skip to main content

How do you crown the best smartphone camera? The experts share their secrets

learn how dxomark scores smartphone cameras fugem 7
Image used with permission by copyright holder

They say the best camera is the one you have with you, and most of us are never without our phones.

But how do you know which smartphone has the best camera? Every manufacturer claims to have a great camera, they throw around megapixel counts and aperture sizes, but none of this tells you how good your shots will be in the real world.

Many of us turn to independent photography lab, DxOMark, for in-depth reviews that boil down the camera quality of each flagship smartphone to a single number out of 100. Since the Parisian company began to test mobile phone cameras in 2012, it has been the gold standard. The scores it bestows are trusted enough to be widely referenced in the tech press.

“When Google achieved top marks [for the Pixel], they covered London in ads because they were proud,” Clement Viard, Senior Director, Image Quality Evaluation at DxOMark told Digital Trends.

Collectively we shared around 2.5 trillion photos online last year and 90 percent of them were captured on smartphones, according to a report from Deloitte. That number is going up every year. The Millennial generation is particularly interested in mobile photography, with 24 percent saying it’s the most important function of their smartphone, and 57 percent saying it’s very important, according to Osterman Research.

Little wonder, then that a good DxOMark is so coveted. But how does the firm arrive at these widely quoted scores?

“Today, the differentiator for the phone is probably the software more than the hardware,”

“For smartphones, we always use the auto mode. We take the phone out of the box, turn it on, start the camera app, and take pictures,” explains Viard. “Camera users are used to tweaking settings, but smartphone users expect to be able to just push a button, so the testing has to be completely different.”

Every smartphone camera is subjected to the same battery of tests, broken down into sections that cover things like exposure and contrast, color, and autofocus. Each camera is meticulously tested with a mix of specialized equipment in the lab.

“We also have to get out of the lab and test in the real-world. We go to the same spots around Paris, around the same time of day. We try to find places where the colors will always be the same, where there is dark and light to see how it handles shading.”

They take 1,500 photos with every smartphone camera that’s tested and then break them down and grade them. To account for changes in light and weather, they also snap shots with a reference device for comparison, so they can be sure they’re grading the phone and not the weather. Each photo is cross-checked by at least two experts from the 40-person team. The result is a 100-page report for each review, highlights of which are published on the DxOMark website.

Viard says they’re usually in agreement, but when there’s a difference of opinion the rest of the team will be called in for a debate. There’s no doubt they take the scoring system seriously, but separating opinion from objective fact can be tricky. You’ll often see people saying that Samsung’s Galaxy phones are oversaturated, for example, but some people prefer the way they look. Where is the line on quality?

Image used with permission by copyright holder

“There was a lot discussion in trying to work out what would be the best way to assess color quality,” explains Viard. “We take pictures of a scene with colors and we map colors into a color space that will look at saturation and bias. What we found is that if the distance between the real color and the color from the camera is within an area, then it’s really a matter of taste. Different people can argue for weeks and they’re not going to come to the same conclusion.”

What DxOMark does in this situation is to stop factoring it into the score beyond a certain point. It does the same thing at the bottom end of the scale. There’s a cut-off point where a photo is so bad that you’ll discard it anyway.

Because DxOMark also offers consulting services and specialized equipment to help manufacturers to improve their camera performance, it has been subject to occasional claims of bias. The idea being that manufacturers who engage its consulting services will get a better score.

“The exact score calculation is confidential and the reason we keep it confidential is because we don’t want manufacturers trying to optimize for the score,” says Viard. “We are very aware that the way we design the score formula may influence some choices companies make, but the testing protocol will be exactly the same for every single phone, with all the same objective metrics, real world results, and human grading.”

DxOMark recently updated its testing protocol for the first time, to add two new categories to every score: Zoom and bokeh. The new protocol also places greater emphasis on low light conditions, and DxOMark introduced more motion into its testing, to see how cameras perform when both the subject and photographer are moving. This is all intended to better reflect what consumers want and the real-world conditions we take photos in.

Image used with permission by copyright holder

The new testing protocol has been a long time in the making. DxOMark is keen to ensure that it only tests features that really matter to people. It considered the impact of 3D capabilities a few years ago, but decided it was a fad. It’s already testing front-facing cameras with the rise in popularity of selfies, and watching 360-degree camera technology very carefully, but it will be a while, if ever, before these make into the score.

“We always try to focus on usage rather than technology,” says Viard.

Apple’s iPhone 7 Plus bokeh effect, where the subject is in sharp focus and the background is blurred, is a good example, as it sparked a demand for bokeh and got everyone talking about it.

“There are many ways of creating the bokeh effect,” Etienne Knauer, Senior VP, Sales and Marketing, Business Solutions, DxOMark explained to Digital Trends. “But you require at least two different images, so with a single camera, you don’t know it, but actually you’re taking multiple images, even if it’s from the same sensor, or they use a dual camera. You need two to make an estimation of the depth and then software blends them.”

Under the old test protocol, there was no distinction between the single-lens iPhone 7 and the dual-lens iPhone 7 Plus, but the introduction of zoom and bokeh, enabled the 7 Plus to creep up the chart with a score of 88, while the iPhone 7 scored 85. The rescored Google Pixel managed 90 on the new scale, putting it joint top with the HTC U11 , but there are some very big phones on the horizon that might challenge the current champions.

“We only test the flagships and the very best phones,” says Viard. “We look at the specs and what the manufacturer is saying to find the phone where they’ve put most emphasis on photography and that’s the one we will pick.”

Sadly, DxOMark lacks the resources to test every phone that comes out, though Viard says they’d like to. There’s no doubt that the field has widened in the last couple of years.

“For the first few years Apple, Nokia, Sony, or Samsung was always the best. Today everybody is putting emphasis on the camera. HTC, Huawei, OnePlus, and Google have made huge improvements to the camera, so the number of companies we test has grown.”

If you’re into smartphone cameras, you’ll know that Sony makes most of the sensors that the flagships use, yet its own phones don’t tend to score as highly as other handsets with the same sensor.

“Today, the differentiator for the phone is probably the software more than the hardware,” says Viard. “Camera tuning makes such a big difference. You can have two manufacturers with exactly the same hardware, even the same chipsets, but the image quality is very different because of the algorithm.”

Even with the enormous processing power smartphone cameras now possess and all the clever software tricks, there are some things a DSLR does better. DxOMark’s bokeh test, for example, is designed to catch out these tricks. It’s a face with triangles that go out to a fine point, wires on the same plain, and lots of details and broken images in the background, to see if the camera gets confused and blurs the wrong parts, which it invariably does.

“On a DSLR, if it’s on the same plain, it will have the same sharpness,” says Viard. “Smartphone cameras have a long road before they reach DSLR level, but I think the gap is closing, the progress has been huge in the last five years.”

All the major manufacturers are investing serious resources into improving camera technology, trying to make it easier for average people to capture perfect shots. That’s great news for us, because at the end of the day, mobile photography is all about capturing precious memories. If you want to know which smartphones will give you the best chance of catching those spontaneous moments, then it’s always worth checking out what DxOMark has to say.

Simon Hill
Former Digital Trends Contributor
Simon Hill is an experienced technology journalist and editor who loves all things tech. He is currently the Associate Mobile…
The best Prime Day foldable phone deals of 2024: Samsung, Google, Motorola
A person folding up the Samsung Galaxy Z Flip 5.

Prime Day deals are underway, with plenty of great offers going on right now. In particular, if you’re looking for excellent Prime Day smartphone deals, you’re in luck. For those looking for the deepest discounts and the most stylish and modern designs, how about the foldable phone deals currently going on? We’ve tracked them all down and included some info on how to choose the right foldable phone for you this Prime Day. Here’s everything you need to know so you can score a great new phone for less than usual. These deals cross over with Motorola Prime Day deals, Samsung Prime Day deals, and Google Pixel Prime Day deals.
Best Samsung foldable phone Prime Day deals

In the foldable phone world, Samsung is best known for its Samsung Galaxy Z Fold 5 and Samsung Galaxy Z Flip 5 phones, with both proving very popular for anyone checking out the best folding phones. Makers of some seriously premium devices, here’s the best Samsung foldable phone Prime Day deals right now so you can save some cash on a phone that you're going to love for a long time to come.
Best Google foldable phone Prime Day deals

Read more
Best Garmin Watch Prime Day deals: Nearly every model is on sale
Three Forerunner 165 watches stacked on a forest floor.

There are some great Prime Day deals around now that we've made it to the big sales event itself. In particular, if you've had a close eye on the Prime Day smartwatch deals unfolding, you're going to love how many different Garmin watch models are discounted right now. If you're looking to treat yourself to one of the best running and general fitness brands around, take a look below at all the options around right now. We also have some key buying advice on what to think about before you commit to any of these. All are some seriously good smartwatch deals.
Best Garmin Watch Prime Day deal
Garmin Forerunner 55 -- $186, was $200

For runners, the Garmin Forerunner 55 is an easy-to-use wearable device that's also affordable -- and even cheaper from this year's Garmin watch Prime Day deals. The smartwatch will monitor your heart rate while you're running, and it has built-in GPS so that you'll be able to track the distance you've run, the path that you took, and your speed for the session. The Garmin Forerunner 55's PacePro feature offers GPS-based guidance for certain courses and distances to help you plan your running strategy, and there are other helpful training tools that you can take advantage of such as race time predictions and finish time estimates.

Read more
This Apple Watch Series 9 deal is too good to pass up
Apps on the Apple Watch Series 9's screen.

It’s never been easier to help keep track of your health and well-being thanks to smartwatches, and the Apple Watch Series 9 is one of the best on the market at doing just that. Whether you’re looking to upgrade your older Apple Watch model or getting your first smartwatch to go with your iPhone, it’s a great time to pick one up. And right now, you can get the latest , which is $100 off the normal price of $400.

The Apple Watch Series 9 is, simply put, one of the best smartwatches you can get. Though the design of the basic Apple Watch has remained unchanged over the years, it’s comfortable to wear and gives you a ton of options for accessorizing with all of the straps and cases on the market. The large always-on display allows you to see all of your important information in a single glance, and there are a ton of useful or fun watch faces to choose from.

Read more