Skip to main content

Photorealistic A.I. tool can fill in gaps in images, including faces

Image used with permission by copyright holder

You only need to go check out the latest Hollywood blockbuster or pick up a new AAA game title to be reminded that computer graphics can be used to create some dazzling otherworldly images when called for. But some of the most impressive examples of machine-generated images aren’t necessarily alien landscapes or giant monsters, they’re image modifications that we don’t even notice.

That’s the case with a new A.I. demonstration created by computer scientists in China. A collaboration between Sun Yat-sen University in Guangzhou and Beijing’s Microsoft Research lab, they’ve developed a smart artificial intelligence which can be used to accurately fill in blank areas in an image: Whether that’s a missing face or the front of a building.

Called inpainting, the technique uses deep learning technology to fill these spaces either by copying image patches on the remainder of the picture, or by generating new areas that look convincingly accurate. The tool, which is referred to by its creators as PEN-Net (Pyramid-context ENcoder Network), does this image restoration by “encoding contextual semantics from full-resolution input and decoding the learned semantic features back into images.” The resulting Attention Transfer Network (ATN) images are not only impressively realistic, but the tool is also very quick to learn.

“[In this work, we proposed] a deep generative model for high-quality image inpainting tasks,” Yanhong Zeng, a lead author on the project, who is associated with both Sun Yat-sen University’s School of Data and Computer Science and Key Laboratory of Machine Intelligence and Advanced Computing, told Digital Trends. “Our model fills missing regions from deep to shallow at all levels, based on a cross-layer attention mechanism, so that both structure and texture coherence can be ensured in inpainting results. We are excited to see that our model is capable of generating clearer textures and more reasonable structures than previous works.”

As Zeng notes, this isn’t the first time researchers have developed tools to carry out inpainting. However, the team’s PEN-Net system demonstrates impressive results next to classical method PatchMatch and even other state-of-the-art approaches.

“Image inpainting has a wide range of applications in our daily life,” Zeng continued. “We are now planning to apply our technology in image editing — especially for object removal [and] old photo restoration.”

A paper describing the work, titled “Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting,” is available to read on preprint paper repository Arxiv.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
The funny formula: Why machine-generated humor is the holy grail of A.I.
microphone in a bar

In "The Outrageous Okona," the fourth episode of the second season of Star Trek: The Next Generation, the Enterprise's resident android Data attempts to learn the one skill it has previously been unable to master: Humor. Visiting the ship’s Holodeck, Data takes lessons from a holographic comedian to try and understand the business of making funny.

While the worlds of Star Trek and the real world can be far apart at times, this plotline rings true for machine intelligence here on Earth. Put simply, getting an A.I. to understand humor and then to generate its own jokes turns out to be extraordinarily tough.

Read more
Nvidia’s latest A.I. results prove that ARM is ready for the data center
Jensen Huang at GTX 2020.

Nvidia just published its latest MLPerf benchmark results, and they have are some big implications for the future of computing. In addition to maintaining a lead over other A.I. hardware -- which Nvidia has claimed for the last three batches of results -- the company showcased the power of ARM-based systems in the data center, with results nearly matching traditional x86 systems.

In the six tests MLPerf includes, ARM-based systems came within a few percentage points of x86 systems, with both using Nvidia A100 A.I. graphics cards. In one of the tests, the ARM-based system actually beat the x86 one, showcasing the advancements made in deploying different instruction sets in A.I. applications.

Read more
Here’s what a trend-analyzing A.I. thinks will be the next big thing in tech
brain network on veins illustration

Virtual and augmented reality. 3D printing. Natural language processing. Deep learning. The smart home. Driverless vehicles. Biometric technology. Genetically modified organisms. Brain-computer interfaces.

These, in descending order, are the top 10 most-invested-in emerging technologies in the United States, as ranked by number of deals. If you want to get a sense of which technologies will be shaping our future in the years to come, this probably isn’t a bad starting point.

Read more