Why is the Pixel’s camera so good? Google explains it, and what’s next

Best products 2016 pixel
Last year, DxO Mark named the camera inside the Google Pixel the best smartphone camera yet — and new insight from Alphabet is showcasing just where the tech behind that camera camera from, and where it’s headed next. In an article for the Graduate Series exploring the tech that comes out of the X research division, Google says the camera tech inside Pixel was actually initially intended for Google Glass.

The tech, called Gcam, was first sparked in 2011 when researchers began looking for a high-resolution camera that could still fit inside a pair of eyeglass frames for Google Glass. Since adding a giant camera on the side of the glasses was out of the question, the team instead started looking at computational photography.

Instead of taking a single image on a large high resolution sensor, Gcam takes several images on a low-resolution sensor and, by merging them together with software, creates a high resolution image — or at least an image that can compete with the typical smartphone camera. The team, lead by Stanford Computer Science faculty Marc Levoy, called it image fusion and the feature launched in Google Glass in 2013.

Creating a small camera didn’t just introduce problems in resolution, however — a smaller lens captures less light, so photos from the lens small enough to hide in Google Glass was also a pretty poor low light performer. Merging the photos helped correct that. But the team next looked at getting more from the tiny camera using high dynamic range, a technique merging multiple images at different exposure levels together to create a wider range of light and detail. HDR+ then launched as an Android camera app for Nexus 5 (and later 6) in 2014.

The computational photography behind the Google Glass camera and HDR+ is now inside the Google Pixel, as well as the Google Photos app, YouTube, and Jump, a virtual reality device. The feature makes the lens blur setting inside Google Photos possible while the same program is piecing together 360 videos for Jump.

Levoy says that it took five years to get the Gcam software right before it launched in the Pixel smartphone. Since the system relies heavily on software, when users started complaining of lens flares, the team launched a firmware update for the software to automatically detect and remove it.

So what’s next for the tiny camera that started in Google Glass and now covers multiple products? The software-focused camera system could be getting a boost based on artificial intelligence. “One direction that we’re pushing is machine learning,” Levan said. “There’s lots of possibilities for creative things that actually change the look and feel of what you’re looking at. That could mean simple things like creating a training set to come up with a better white balance. Or what’s the right thing we could do with the background  —  should we blur it out, should we darken it, lighten it, stylize it? We’re at the best place in the world in terms of machine learning, so it’s a real opportunity to merge the creative world with the world of computational photography.”

Whatever’s in store for the next variation, the success of the Pixel likely sets that bar pretty high.

Mobile

Midrange phones can’t do A.I., but MediaTek’s P90 chip aims to change that

MediaTek has announced the Helio P90 mobile processor, which it says will bring the best A.I. features we see on high-end smartphones, to the mid-range. We spoke to the company about the chip.
Mobile

5G’s arrival is transforming tech. Here’s everything you need to know to keep up

It has been years in the making, but 5G is finally becoming a reality. While 5G coverage is still extremely limited, expect to see it expand in 2019. Not sure what 5G even is? Here's everything you need to know.
Mobile

The Galaxy S10 may be announced before MWC, sell for up to $1,750

While we still may be months away from an announcement, there's no doubt about it: Samsung is working hard on its successor to the Galaxy S9. Here's everything we know about the upcoming Samsung Galaxy S10.
Mobile

Honor to out-megapixel the competition with 48MP camera on upcoming View 20

After its phenomenal success with the View 10 in 2018, it looks like Honor is getting ready up the ante with its forthcoming Honor View 20. Here's everything we know about it so far.
Photography

The best mirrorless cameras pack all the power of a DSLR, minus the bulk

Mirrorless cameras offer a lot of photography firepower, inside a compact body. Explore the best mirrorless cameras, from the pro-level to the beginner-friendly shooters, in this guide.
Photography

Photography News: Startup redesigns tripod heads ‘inside out’ for more flexibility

Well, this doesn't look like the ball heads that we've seen before. Instead of designing a tripod ball head with a small cutout, the Colorado Tripod Company created one with most of the ball exposed, allowing for more possible angles.
Photography

MIT science photographer isn’t an artist, but her work could fill galleries

Felice Frankel is an award-winning photographer, but she doesn't consider herself an artist. As a science photographer, she has been helping researchers better communicate their ideas for nearly three decades with eye-catching imagery.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Folding canoes and ultra-fast water filters

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Photography

Leica targets street photographers with a pricey camera bundle

Described by the camera company as "your perfect companion in the city," Leica's Street Kit comprises a Leica CL camera body, a 23mm (35mm full-frame equivalent) F2 lens, batteries, a handgrip, and a black leather carrying strap.
Product Review

Meet Z6, the breakout star in Nikon's new mirrorless lineup

The Nikon Z6 is the sibling to the new mirrorless Z7 -- but for some photographers, the cheaper Z6 may be the better option. Read where the $2,000 camera beats the $3,400 one (and where it doesn’t) in our Nikon Z6 review.
Social Media

Snapchat facial recognition could soon power a new portrait mode, code suggests

Digging into Snapchat's code suggests a handful of upcoming camera features, including a portrait mode. The feature appears to use facial recognition A.I. to blur the background. The code also suggests an updated camera interface.
Social Media

#ThrowbackThursday is only the start: Instagram hashtags for every day of the week

Not getting your hashtag fill with #ThrowbackThursday or #ManCrushMonday? Here's a list of some of the more popular Instagram hashtags, so you can outfit your next post with the proper tag, regardless of what day it is.
Photography

From DSLRs to mirrorless, these are the best cameras you can buy right now

From entry-level models to full-frame flagships, many cameras take great photos and video. The best digital cameras, however, push the industry forward with innovative sensors and improved usability, among other things. Here are our…
Photography

Photographers can now customize the layout of Lightroom Classic controls

Tired of scrolling past Lightroom tools that you don't use? Adobe Lightroom Classic now allows users to reorganize the Develop panel. The update comes along with new sharing options in Lightroom CC, and updates to the mobile Lightroom app.