Smartphones suck at low-light photos, but the right software could fix that

florian kainz low light smartphone experiment raw 2016 06 20 21 17 45 471 header
Florian Kainz / Google Research
When the average person accepts a dare, the result may range anywhere from a few stitches to some embarrassing photos. But when a Google software engineer accepts a challenge, the result might just be a solution that solves a common problem.

After Florian Kainz, of Google Research’s Gcam team, showed his co-workers a night landscape he’d shot with a DSLR, he was challenged to take the same photo, but with a smartphone. The result? A peek at a shooting and editing process, which he documented in a Google blog post, could eventually wind up in future Google smartphones, like the Pixel.

Kainz took inspiration from the HDR+ mode that combines multiple smartphone photos for better quality, as well as SeeInTheDark, an experimental app that produces low-light images by merging photos and dropping the resolution to a single megapixel. Using similar multiple-shot techniques, Kainz set out to see if he could shoot a photo in the dark but without the noisy images from SeeInTheDark or the limitations of HDR+.

He decided to use a similar bracketing method of merging multiple photos together to get better – and brighter – results. Unlike the automatic HDR+ feature, he merged long exposures instead of images taken at a tenth of a second, using the smartphone’s longest available shutter speed (four seconds for the Nexus 6P and two seconds for the Pixel). The longer exposures would be brighter and merging multiple brighter photos would produce better results, he theorized.

But first, he had to tackle the challenges all cameras face in the dark: focusing. Cameras need light to focus, which means even DSLR users need to switch to manual focus to shoot at night. Kainz then had to program an app that allows the camera’s focus to be manually set. For landscape photos, setting the camera’s focus at infinity worked for most of the subjects he shot, he wrote. The app also allowed him to control the shutter speed and ISO, as well as shooting in DNG, a type of digital negative with more control over the editing process.

Manual camera apps aren’t anything new, but Kainz’s experimental app also shot up to 64 single photos with one press of the shutter button – over six times the maximum ten frames the HDR+ will capture. To test the app, he shot at a number of locations during the light of a full moon as well as nights with no moon.

Kainz was able to successfully shoot low-light shots with detail not possible before, including capturing the Milky Way.

Those shots, with as many as 64 single photos including some black exposures with tape over the lens, were then taken into a desktop photo editing program and merged to eliminate the grain of high ISO. In scenarios where the ground or sky wasn’t sharp, he combined the images using layer masks to determine which parts of which photo showed in the final image.

The result? While the resolution still isn’t the same as a DSLR, Kainz was able to successfully shoot low-light shots with detail not possible before, including capturing the Milky Way.

While the process still requires a tripod, Kainz says an app that could handle the merging of those 64 photos could make low-light photography from a smartphone more accessible, even for users that have no idea how to merge 64 photos or how to use layer masks. “Trying to find out if phone cameras might be suitable for outdoor nighttime photography was a fun experiment, and clearly the result is yes, they are,” he wrote. “However, arriving at the final images required a lot of careful post-processing on a desktop computer, and the procedure is too cumbersome for all but the most dedicated cellphone photographers. However, with the right software a phone should be able to process the images internally, and if steps such as painting layer masks by hand can be eliminated, it might be possible to do point-and-shoot photography in very low-light conditions.”

Kainz doesn’t say whether Google is working on an app that could do all the heavy processing without a desktop computer, but he says the software might just be possible. To see the rest of the shots from the experiment, check out Kainz’ Google Photos album.

Product Review

Equal parts tool and toy, the Lensbaby Edge 35 bucks photographic tradition

The Lensbaby Edge 35, part of the Composer Pro optic swap system, creates tilt-shift-like blur without the tilt-shift price. Made for photographers who want find tradition boring, it opens up new ways to work with blur.
Emerging Tech

Gravitational forces at heart of Milky Way shaped this star cluster like a comet

Hubble has captured the stunning Messier 62 cluster. The cluster is warped, with a long tail which stretches out to form a shape like a comet. It is thought this distortion is due to Messier 62's proximity to the center of the galaxy.
Photography

Light on price but rich on features, these are the best cameras for students

Need pro-level features on a budget? The best cameras for students mix advanced features with a more palatable price point. From $2K entry level full frame cameras to $600 budget picks, here are five of the best cameras for students.
Gaming

The Last of Us Part 2 finishes capturing its main characters’ performances

Calling it the most "ambitious cinematic shoot" the team has ever done, Naughty Dog revealed that the motion-capture performances for the main characters in The Last of Us Part II are finished.
Emerging Tech

Watch the fearsome DroneHunter X3 pluck rogue UAVs out of the sky

How do you stop enemy drones in their tracks? DroneHunter X3 is a new autonomous anti-drone technology which outruns and then captures rogue drones in midair. Check it out in action.
Deals

The best budget-friendly GoPro alternatives that won’t leave you broke

Cold weather is here, and a good action camera is the perfect way to record all your adventures. You don't need to shell out the big bucks for a GoPro: Check out these great GoPro alternatives, including some 4K cameras, that won’t leave…
Photography

The Black Eye Pro Cinema Wide G4 is a knockout lens for any smartphone

Where cheaper wide-angle accessory lenses add distortion, and costlier models don't always justify their higher prices, the Black Eye Pro Cinema Wide G4 offers a valuable balance of modest price and high quality optics.
Photography

Etch-A-Snap camera puts a modern spin on one of your favorite childhood toys

Can't draw on an Etch A Sketch? Snap a photo with the Etch-A-Snap and the camera will draw out the scene for you. The weirdly cool camera designed by Martin Fitzpatrick replaces the usual LCD screen with an old-school Etch A Sketch.
Photography

Family feud: Huawei P30 Pro vs. P20 Pro vs. Mate 20 Pro camera shootout

The Huawei P30 Pro's camera has an amazing zoom mode and low light capabilities. But take these away, and how does it compare when facing its sibling phones, the P20 Pro and Mate 20 Pro, taking regular photos?
Photography

Nikon Z 7 vs. Sony A7R III: High-res mirrorless cameras compared

The Nikon Z 7 and Sony A7R III both have over 40 megapixels, but which one comes out on top? With similar image quality, the answer comes down to speed, autofocus, battery life, and design.
Deals

The Canon EOS Rebel T6 DSLR camera gets a steep price cut at Walmart

Modern smartphones can snap pretty impressive pics, but if you want pro-quality photos, you need a dedicated digital camera. The Canon EOS Rebel T6 is one of the best entry-level DSLR cameras on the market, and it’s on sale right now for…
Photography

Panasonic Lumix S1R vs. Nikon Z 7: When megapixels matter, which do you choose?

The 47-megapixels Lumix S1R and 46-megapixel Nikon Z 7 are the two highest-resolution, full-frame mirrorless cameras on the market. The S1R features a high-resolution mode that can take 187MP images, but the Nikon is lighter and cheaper.
Photography

Sweet 16: Wacom’s Cintiq 16 pen display makes retouching photos a breeze

Wacom’s Cintiq pen displays are usually reserved for the pros (or wealthy enthusiasts), but the new Cintiq 16 brings screen and stylus editing to an approachable price. Does it cut too much to get there?
Mobile

China bans selfies at gigantic Aperture Spherical Telescope

You can't take a selfie with the world's largest single-dish radio telescope anymore, as the Chinese government has banned everything from smartphones to digital cameras in the surrounding 5-kilometer area.