Using this A.I.-based healing brush, repairing an image is no biggie

Photoshop’s healing brush can use surrounding pixels to repair an image or remove an object — but what if the surrounding pixels don’t have enough data to fill in those holes? Researchers from Nvidia recently used artificial intelligence to help fill in those gaps to create a tool similar to the healing brush tool that is able to intelligently fill in missing pieces, a technique called image inpainting for irregular holes using partial convolutions — here’s hoping Nvidia comes up with a nickname before making the tool widely accessible.

Nvidia isn’t the first to try to reinvent the healing brush using A.I., but the researchers say that earlier attempts left artifacts and blur. Rather than using full convolutional filter responses to those holes, Nvidia instead created a partial convolution method — in other words, the software creates a layer that renormalizes the pixels to create a more normal looking image without those artifacts. The new program also uses an automatically generated layer masks.

Unlike the traditional healing brush tool that only uses surrounding pixels to determine what to fill in the gap with, Nvidia trained its tool using three different sets of images, resulting in thousands of images to train the tool with. The researchers randomly applied masks to intentionally remove sections of images to those groups of photos. By showing the computer both the before and after, the program could learn how to fill in some of those gaps. “Our model can robustly handle holes of any shape, size location or distance from the image borders,” the researchers wrote. “Further, our performance does not deteriorate catastrophically as holes increase in size.”

While more robust than previous attempts, the researchers said that the tool struggles with the largest holes and images without a lot of structure.

The research could bring some significant changes to photo editing if the tool makes its way into an image editor. Nvidia’s program, for example, could replace an eye in an old, damaged portrait, even without the other eye to replicate. The resulting eye, of course, isn’t the same eye — in one example, the replaced eye is an entirely different color. In the demonstration video, the program also appears to give a young woman and an older gentleman the same pair of eyes. 

While giving a person different features using the database of images should raise ethical considerations for portraits, the concept could speed up the process of removing distractions from other types of images. The tool could potentially help remove objects in photographs, such as removing power lines or signs from the background, with more accuracy than current tools.


Nvidia’s A.I. Playground lets you edit photos, experience deep learning research

Nvidia is making it easier to access information on deep learning research. It has launched an online space with three demos for image editing, styling, as well as photorealistic image synthesis. 

G-Sync and FreeSync can make your games look better, but which is best?

There are some subtle differences between the two adaptive refresh technology offerings, and they affect cost, performance, and compatibility. Nvidia may have released it's feature first, but in recent years AMD has stepped up to the plate…

Nvidia’s smart paintbrush tool turns simple doodles into works of art

Nvidia has announced a new deep-learning-enhanced drawing app that can turn your doodles and sketches into highly realistic images. With Nvidia's GauGAN, simple sketches can be automatically embellished with more vivid details.
Smart Home

Is your Keurig making gross coffee? Might be time for a cleaning

No one likes a dirty, scaled, or smelly Keurig, but how are you supposed to clean them? Before you throw yours out the window, here is a quick guide on cleaning your machine out thoroughly.
Social Media

Twitter takes a cue from Instagram and Snapchat with new quick-swipe camera

Twitter is giving the "what's happening" treatment to photos and video by allowing users to access the in-app camera fast enough to catch and share the moment. The new Twitter camera is now accessible with a swipe.

The Lensbaby Composer Pro II with Edge 35 mimics tilt-shift blur for less cash

Want to create a tilt-shift image on a budget? The new Lensbaby Composer II with Edge 35 mimics the look of a tilt-shift lens for under $500. The new Edge 35 optic is part of the Composer Pro II optics system.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Write music with your voice, make homemade cheese

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!

Loupedeck Plus can now edit video, audio with Final Cut Pro

The list of Loupedeck Plus-compatible software is growing. The photo-editing console now works with Final Cut Pro and Adobe Audition for video and audio editing. The controls can be configured to be used on either platform.

Photography News: Taking a smartphone photo probably saved this guy’s life

A man was snapping a photo in Australia when the smartphone stopped an arrow shot at his face. In this week's photography news, see Canon's plans for a stabilized mirrorless, Hasselblad's newest accessories, Samyang's latest lens, and more.

The best budget-friendly GoPro alternatives that won’t leave you broke

Cold weather is here, and a good action camera is the perfect way to record all your adventures. You don't need to shell out the big bucks for a GoPro: Check out these great GoPro alternatives, including some 4K cameras, that won’t leave…

Paper designs digitize in real time using an Illustrator-connected paper tablet

Love graphic design, but prefer the feel of real paper? The new Moleskine Paper Tablet - Creative Cloud Connected syncs with Adobe Illustrator in real time, turning paper sketches into digital drawings.

Insta-checkout? New Instagram service lets you shop without leaving the platform

Shopping on Instagram no longer means leaving the platform to checkout in a web browser. Instagram checkout launched in beta today with a handful of retailers, allowing users to checkout without leaving the app.

Make the most of your toner with our five favorite color laser printers

Color laser printers have improved dramatically over the years, and today's models offer both blazing print speeds and great image quality. Here are our favorite color laser printers, from massive all-in-ones to smaller budget options.

Sony’s latest sensor is stacked, backlit, and equipped with a global shutter

Say this five times fast: Backlit, stacked, global shutter sensor. Sony managed to cram all three technologies into one sensor. The result is a high-speed sensor with a higher resolution without sacrificing low-light quality.