Programmer trains artificial intelligence to draw faces from text descriptions

Programmer Animesh Karnewar wanted to know how characters described in books would appear in reality, so he turned to artificial intelligence to see if it could properly render these fictional people. Called T2F, the research project uses a generative adversarial network (GAN) to encode text and synthesize facial images.

Simply put, a GAN consists of two neural networks that argue with each other to produce the best results. For example, the job of network No. 1 is to fool network No. 2 into believing a rendered image is a real photograph while network No. 2 sets out to prove the alleged photo is just a rendered image. This back-and-forth process fine-tunes the rendering process until network No. 2 is eventually fooled.

Karnewar started the project using a dataset called Face2Text provided by researchers at the University of Copenhagen, which contains natural language descriptions for 400 random images.

“The descriptions are cleaned to remove reluctant and irrelevant captions provided for the people in the images,” he writes. “Some of the descriptions not only describe the facial features, but also provide some implied information from the pictures.”

While the results stemming from Karnewar’s T2F project aren’t exactly photorealistic, it’s a start. The video embedded above shows a time-lapsed view of how the GAN was trained to render illustrations from text, starting with solid blocks of color and ending with rough but identifiable pixilated renderings.

“I found that the generated samples at higher resolutions (32 x 32 and 64 x 64) has more background noise compared to the samples generated at lower resolutions,” Karnewar explains. “I perceive it due to the insufficient amount of data (only 400 images).”

The technique used to train the adversarial networks is called “Progressive Growing of GANs,” which improves quality and stability over time. As the video shows, the image generator starts from an extremely low resolution. New layers are slowly introduced into the model, increasing the details as the training progresses over time.

“The Progressive Growing of GANs is a phenomenal technique for training GANs faster and in a more stable manner,” he adds. “This can be coupled with various novel contributions from other papers.”

artificial intelligence trained to render faces from text ai facial rendering

In a provided example, the text description illustrates a woman in her late 20s with long brown hair swiped over to one side, gentle facial features and no make-up. She’s “casual” and “relaxed.” Another description illustrates a man in his 40s with an elongated face, a prominent nose, brown eyes, a receding hairline and a short mustache. Although the end results are extremely pixelated, the final renders show great progress in how A.I. can generate faces from scratch.

Karnewar says he plans to scale out the project to integrate additional datasets such as Flicker8K and Coco captions. Eventually, T2F could be used in the law enforcement field to identify victims and/or criminals based on text descriptions, among other applications. He’s open to suggestions and contributions to the project.

To access the code and contribute, head to Karnewar’s repository on Github here.

Emerging Tech

Short film celebrates New Yorker’s amazing robot costumes

New York City resident Peter Kokis creates stunning robot costumes out of household trash. His designs are huge, heavy, and extremely intricate, and never fail to turn heads when he's out and about.
Mobile

Unleash your inner artist with the best drawing apps for the iPad Pro

The best drawing apps for the iPad Pro turn Apple's robust tablet into a canvas worthy of a true artist. Here are 20 of our current favorites, whether you're looking to keep things simple or take your art to the next level.
Emerging Tech

A.I. can generate pictures of a finished meal based on just a written recipe

Feeling hungry? Researchers from Tel-Aviv University in Israel have created an A.I. that's able to generate complete photos of meals, based only on reading the written recipes. Here's how.
Gaming

Here's where Xur is and what he has for wares this week in 'Destiny 2: Forsaken'

The weekly vendor in Destiny 2: Forsaken always brings Exotic weapons and armor, some of the toughest loot to find in the game. Here's everything you need to know to track down Xur: Where he is, when he shows up, and what he's stocking.
Computing

Delete tracking cookies from your system by following these quick steps

Cookies are useful when it comes to saving your login credentials and other data, but they can also be used by advertisers to track your browsing habits across multiple sites. Here's how to clear cookies in the major browsers.
Computing

Convert your PDFs into convenient Word documents with Adobe or a free option

PDF files are great, but few document types are as malleable as those specific to Microsoft Word. Here's how to convert a PDF file into a Word document, whether you prefer to use Adobe's software suite or a freemium alternative.
Computing

Nvidia’s next midrange card might be a GTX 1660 Ti, rumors suggest

Nvidia may be working on a non-RTX Turing graphics card called the 1660 Ti. Rumors suggest it will have around 20 percent fewer CUDA cores than the RTX 2060 and will lack ray tracing support.
Deals

From Samsung to HP, here are the best cheap Chromebook deals right now

Whether you want a compact laptop to enjoy some entertainment on the go, or you need a no-nonsense machine for school or work, we've smoked out the best cheap Chromebook deals -- from full-sized laptops to 2-in-1 convertibles -- with most…
Computing

Chromebook 13 vs. Google Pixelbook: Acer model takes on the king

Acer's Chromebook 13 is throwing tons of speed at the Chrome OS market, to go with a midrange build and traditional clamshell design. Is that enough to challenge the Google Pixelbook?
Computing

Data breach compromises 773 million records, 21 million passwords

A security researcher was alerted to a collection of breached data that included more than 773 million compromised records. After digging deeper, the breach was revealed to contain more than 21 million passwords.
Computing

Dell teases new XPS laptop with Intel’s 10th-gen Ice Lake for summer 2019

After teasing a mysterious Ice Lake-powered XPS laptop during Intel's keynote, Dell confirmed that it will announce a new 10th-generation Ice Lake-powered XPS laptop this year. The new XPS notebook could debut as early as summer.
Computing

Faster new PCIe 5.0 standard leapfrogs the best feature of AMD’s Ryzen 3

PCIe 5.0 will bring even faster data transfers, but it may only be found on HPCs and servers initially. The standard is four times faster than your current PC at transferring data, and new devices could appear later this year.
Deals

From Chromebooks to MacBooks, here are the best laptop deals for January 2019

Whether you need a new laptop for school or work or you're just doing some post-holiday shopping, we've got you covered: These are the best laptop deals going right now, from discounted MacBooks to on-the-go gaming PCs.
Computing

Keep your laptop battery in tip-top condition with these handy tips

Learn how to care for your laptop's battery, how it works, and what you can do to make sure yours last for years and retains its charge. Check out our handy guide for valuable tips, no matter what type of laptop you have.