Programmer trains artificial intelligence to draw faces from text descriptions

Programmer Animesh Karnewar wanted to know how characters described in books would appear in reality, so he turned to artificial intelligence to see if it could properly render these fictional people. Called T2F, the research project uses a generative adversarial network (GAN) to encode text and synthesize facial images.

Simply put, a GAN consists of two neural networks that argue with each other to produce the best results. For example, the job of network No. 1 is to fool network No. 2 into believing a rendered image is a real photograph while network No. 2 sets out to prove the alleged photo is just a rendered image. This back-and-forth process fine-tunes the rendering process until network No. 2 is eventually fooled.

Karnewar started the project using a dataset called Face2Text provided by researchers at the University of Copenhagen, which contains natural language descriptions for 400 random images.

“The descriptions are cleaned to remove reluctant and irrelevant captions provided for the people in the images,” he writes. “Some of the descriptions not only describe the facial features, but also provide some implied information from the pictures.”

While the results stemming from Karnewar’s T2F project aren’t exactly photorealistic, it’s a start. The video embedded above shows a time-lapsed view of how the GAN was trained to render illustrations from text, starting with solid blocks of color and ending with rough but identifiable pixilated renderings.

“I found that the generated samples at higher resolutions (32 x 32 and 64 x 64) has more background noise compared to the samples generated at lower resolutions,” Karnewar explains. “I perceive it due to the insufficient amount of data (only 400 images).”

The technique used to train the adversarial networks is called “Progressive Growing of GANs,” which improves quality and stability over time. As the video shows, the image generator starts from an extremely low resolution. New layers are slowly introduced into the model, increasing the details as the training progresses over time.

“The Progressive Growing of GANs is a phenomenal technique for training GANs faster and in a more stable manner,” he adds. “This can be coupled with various novel contributions from other papers.”

artificial intelligence trained to render faces from text ai facial rendering

In a provided example, the text description illustrates a woman in her late 20s with long brown hair swiped over to one side, gentle facial features and no make-up. She’s “casual” and “relaxed.” Another description illustrates a man in his 40s with an elongated face, a prominent nose, brown eyes, a receding hairline and a short mustache. Although the end results are extremely pixelated, the final renders show great progress in how A.I. can generate faces from scratch.

Karnewar says he plans to scale out the project to integrate additional datasets such as Flicker8K and Coco captions. Eventually, T2F could be used in the law enforcement field to identify victims and/or criminals based on text descriptions, among other applications. He’s open to suggestions and contributions to the project.

To access the code and contribute, head to Karnewar’s repository on Github here.

Emerging Tech

Awesome Tech You Can’t Buy Yet: Robo sidekicks, AC for your bed, and more

Check out our roundup of the best new crowdfunding projects and product announcements that hit the Web this week. You can't buy this stuff yet, but it sure is fun to gawk!
Movies & TV

David Harbour is 'taking risks' with Jim Hopper in 'Stranger Things' season 3

With a sophomore season as strong as its first, Stranger Things is now moving on to season 3. Here's everything we've learned so far about the Netflix series' upcoming third season.
Cars

Tesla board waits for Musk’s plan to go private as Saudi Arabia stays silent

Tesla co-founder and CEO Elon Musk asked more questions than he answered when he declared he could take Tesla private. The money allegedly comes from Saudi Arabia, an oil-rich nation and Tesla investor, but officials have stayed silent.
Gaming

The best PS1 games of all time

Take a stroll down memory lane with the 50 best games ever released for the original PlayStation.
Computing

Apple AR glasses will launch in 2020, says respected industry analyst

Apple AR glasses may be closer to reality than we thought. Here is everything we know so far about the augmented reality system, including the rumored specifications of Apple's Project Mirrorshades.
Social Media

How to use Adobe Spark Post to spice up your social media images

Images are proven to get more likes than plain text -- but only if those images are good. Adobe Spark post is an AI-powered design program for non-designers. Here's how to use it to take your social media feeds to the next level.
Photography

A turn for the better: Loupedeck+ adds custom dials, more to Lightroom console

The Loupedeck+ improves on the original Lightroom console by adding welcome customization options and introducing support for Skylum Aurora HDR. What's even better is that it does this all at an even lower price.
Mobile

Google One subscriptions offer more cloud storage for low prices, other perks

Can't get enough storage on Google Drive, Photos, or Gmail? Google One is the new way to boost your cloud storage. But it's not just about more space -- Google One comes with a loads of benefits.
Computing

Intel serves up ‘Bean Canyon’ NUCs revved with ‘Coffee Lake’ CPUs

Looking for a super-compact PC for streaming media that doesn’t break the bank? Intel updated its NUC family with its new “Bean Canyon” kits. Currently, there are five with a starting price of $300 packing eighth-generation Intel Core…
Deals

Save hundreds with the best MacBook deals for August 2018

If you’re in the market for a new Apple laptop, let us make your work a little easier: We hunted down the best up-to-date MacBook deals available online right now from various retailers.
Computing

Lost without 'Print Screen'? Here's how to take a screenshot on a Chromebook

Chrome OS has a number of built-in screenshot options, and can also be used with Chrome screenshot extensions for added flexibility. You have a lot of options, but learning how to take a screenshot on a Chromebook is easy.
Computing

Gaming on a laptop has never been better. These are your best options

Gaming desktops are powerful, but they tie you down to your desk. For those of us who prefer a more mobile experience, here are the best gaming laptops on the market, ranging from budget machines to maxed-out, wallet-emptying PCs.
Computing

A dead pixel doesn't mean a dead display. Here's how to repair it

Dead pixel got you down? We don't blame you. Check out our guide on how to fix a dead pixel and save yourself that costly screen replacement, or an unwanted trip to your local repair shop.
Computing

Asus claims ‘world’s thinnest’ title with its new Zephyrus S gaming laptop

The Republic of Gamers arm at Asus is claiming “world’s thinnest” with the introduction of its new Zephyrus S gaming laptop measuring just 0.58 inches at its thinnest point. The company also revealed the Strix SCAR II.