Clipping out objects from photos and videos is often a time-consuming process — but Adobe is looking for new ways to change that with deep learning software. Published earlier this month, the research for Deep Image Mapping by Adobe, the Beckman Institute for Advanced Science and Technology and the University of Illinois at Urbana-Champaign aims to improve on software-based clipping solutions.
Researchers have developed a number of different artificially intelligent programs to automatically subtract a background from an image, but most are based on colors. When presented with an image with similar colors such as the greens in a landscape photo, those programs tend to fail, Adobe says. Similar applications also use a very narrow set of data points to train the software, the researchers say, which makes the system biased to only certain types of images.
Adobe’s latest method, however, relies on the structure of the objects in the photograph, not the colors. The program requires the image and a trimap, or a rough sketch of what should be cut from the image that labels the foreground, the background, and a range in between to enable the computer to find the edges.
The program works by learning the structure of alpha mattes, or the “color” channel that contains all the transparencies in an image. Common alpha mattes include hair and fur as well as semi-transparent objects or even optical blur created from a slow shutter speed. The research team used 49,300 images to train the deep learning program and 1,000 images to test its accuracy. By recognizing the structure instead of separating by color, the program has achieved accurate results even in images with similar tones.
Adobe’s process puts the image through two rounds of encoding, the first to find those edges and the second to refine the details for more accurate clips.
While artificially intelligent image clipping methods are still under research, Adobe’s latest look into the process could mean better clipping tools will eventually make their way inside programs like Photoshop and Premiere Pro — and even the possibility of eliminating the traditional green screen if the feature is robust enough.
