Google and MIT’s automatic photo software can edit your shots before you take them

Photo editing software often enhances a smartphone’s limited hardware, but what if those shots could be edited automatically — in real time? That’s the question researchers from the Massachusetts Institute of Technology (MIT) and Google asked when expanding an earlier machine-learning program that automatically improved shots after sending them to the cloud. Presented during this week’s Siggraph digital graphics conference, the new automatic photo software edits so quickly that the user actually sees the results on the screen in real time — before that shot is even taken.

The program is based on earlier MIT research that trained a computer to edit images automatically. Researchers taught that program how to automatically apply specific adjustments by feeding the system five variations of each image, all edited by a different photo editor. After repeating that process a few thousand times, the program learned how to identify and fix common image issues.

The new software is built on the same artificial intelligence platform but speeds up the program to the point where it completes edits in a tenth of the previous time. This allows the live view from the camera to show edits in real time instead of sending the shot to a cloud system for editing. So how did researchers achieve the speed boost? The output of the software actually isn’t an image, but an image formula. Calculating the changes (and only applying them if a photo is taken) speeds up the process.

The original image is also divided into grids, a form of downsizing the image for faster editing without losing pixels in the final image. The program works on each of the divided sections at once, editing hundreds of pixels simultaneously instead of factoring every single pixel.

With those two changes, the program only needs about 100 megabytes to perform each image edit, while without those two modifications the same software needed 12 gigabytes.

The research is a growing trend of looking to computational photography to solve the shortcomings of the hardware that can fit inside of a smartphone. Unlike earlier programs, the automatic photo software from MIT and Google looks to solve one of computational photography’s biggest obstacles, the limited processing power of a mobile device. In the researcher’s experiment, the program applied a high-dynamic range algorithm in real time, boosting the image’s colors and range of light beyond what the hardware itself could achieve.

“This technology has the potential to be very useful for real-time image enhancement on mobile platforms,” said Jon Barron, a Google researcher who worked on the project. “Using machine learning for computational photography is an exciting prospect but is limited by the severe computational and power constraints of mobile phones. This paper may provide us with a way to sidestep these issues and produce new, compelling, real-time photographic experiences without draining your battery or giving you a laggy viewfinder experience.”

Product Review

Canon democratizes full-frame with the EOS RP, but keep your expectations low

At just $1,300, the RP is Canon's least expensive full-frame camera yet, but it was born into a world of high-end, high-cost lenses where it doesn't yet feel at home.
Photography

Luminar’s libraries gain speed, drop need for you to manually import images

Luminar 3 just got a performance boost. Skylum Luminar 3.0.2 has improved speed over December's update, which added the long-promised libraries feature giving editors a Lightroom alternative.
Smart Home

Your office is a mess, and it’s making Marie Kondo cry. Here’s how to tidy it up

Here's how to "Marie Kondo" your office. If you've been inspired to remove clutter and create a minimalistic workspace that makes you happy and helps you focus on what matters, then we have ideas that you'll want to try.
Photography

What’s the difference between Lightroom CC and Lightroom Classic?

Lightroom CC has evolved into a capable photo editor, but is it enough to supplant Lightroom Classic? We took each program for a test drive to compare the two versions and see which is faster, more powerful, and better organized.
Photography

Olympus packs an enormous zoom ability in its latest interchangeable lens

The Olympus Digital ED M.Zuiko 12-200mm F/3.5-6.3 has the widest zoom range of any interchangeable lens with a 16.6x zoom. The lens, which covers a 24-400mm equivalent, is also weather sealed.
Photography

Nikon brings a classic workhorse lens to the Z series with new 24-70mm f/2.8 S

The Nikon Z series finally has a bright zoom available without an adapter. The Nikkor Z 24-70mm F/2.8 S offers new coatings and more customizable controls in a smaller, lighter body than the comparable F-mount lens.
Mobile

OnePlus 6T vs. Honor View 20: We compare the cameras in these ‘flagship killers’

For less than $600, you can buy either the OnePlus 6T or the Honor View 20, two extremely capable smartphones with plenty of exciting features. But which one has the best camera? We found out on a recent trip to France.
Photography

Nikon will bring eye-detection autofocus to the Z6 and Z7 in May

An upcoming firmware update will bring Eye AF to the Nikon Z6 and Z7 -- along with improved autofocus performance in low light. The update will also give the cameras support for the CFexpress format.
Photography

Fujifilm’s X-T30 is a semi-pro, feature-rich camera that’s affordable to boot

Fujifilm's newest mirrorless camera delivers the premium features of the X-T3 without the premium price, giving aspiring enthusiasts a lower-cost option that can still match the image quality of Fuji's flagship.
Photography

Fujifilm XP140 squeezes more durability, low-light ability into a waterproof cam

Fujifilm's waterproof compact can now head even further underwater. The Fujifilm XP140 features several upgrades, including a more durable body, a wider ISO range for low light, and expanded auto modes.
Photography

From f/1.2 primes to the mysterious DS, here are Canon’s upcoming RF lenses

Canon's EOS R mirrorless series will gain six new lenses this year. Canon just shared a list of six lenses under development, including four zooms and two prime lenses. One has a mysterious new feature called Defocus Smoothing.
Photography

From DSLRs to mirrorless, these are the best cameras you can buy right now

From entry-level models to full-frame flagships, many cameras take great photos and video. The best digital cameras, however, push the industry forward with innovative sensors and improved usability, among other things. Here are our…
Photography

Photography news: Wacom’s slimmer pen, Leica’s cinema special edition

In this week's photography news, Wacom launches a new slimmer pen for pro users. Leica's upcoming M10-P is designed for cinema, inside and out, with built-in cinema modes in the updated software.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Grow veggies indoors and shower more efficiently

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!