Skip to main content

Programmer trains artificial intelligence to draw faces from text descriptions

T2F training time lapse

Programmer Animesh Karnewar wanted to know how characters described in books would appear in reality, so he turned to artificial intelligence to see if it could properly render these fictional people. Called T2F, the research project uses a generative adversarial network (GAN) to encode text and synthesize facial images.

Recommended Videos

Simply put, a GAN consists of two neural networks that argue with each other to produce the best results. For example, the job of network No. 1 is to fool network No. 2 into believing a rendered image is a real photograph while network No. 2 sets out to prove the alleged photo is just a rendered image. This back-and-forth process fine-tunes the rendering process until network No. 2 is eventually fooled.

Karnewar started the project using a dataset called Face2Text provided by researchers at the University of Copenhagen, which contains natural language descriptions for 400 random images.

“The descriptions are cleaned to remove reluctant and irrelevant captions provided for the people in the images,” he writes. “Some of the descriptions not only describe the facial features, but also provide some implied information from the pictures.”

While the results stemming from Karnewar’s T2F project aren’t exactly photorealistic, it’s a start. The video embedded above shows a time-lapsed view of how the GAN was trained to render illustrations from text, starting with solid blocks of color and ending with rough but identifiable pixilated renderings.

“I found that the generated samples at higher resolutions (32 x 32 and 64 x 64) has more background noise compared to the samples generated at lower resolutions,” Karnewar explains. “I perceive it due to the insufficient amount of data (only 400 images).”

The technique used to train the adversarial networks is called “Progressive Growing of GANs,” which improves quality and stability over time. As the video shows, the image generator starts from an extremely low resolution. New layers are slowly introduced into the model, increasing the details as the training progresses over time.

“The Progressive Growing of GANs is a phenomenal technique for training GANs faster and in a more stable manner,” he adds. “This can be coupled with various novel contributions from other papers.”

Image used with permission by copyright holder

In a provided example, the text description illustrates a woman in her late 20s with long brown hair swiped over to one side, gentle facial features and no make-up. She’s “casual” and “relaxed.” Another description illustrates a man in his 40s with an elongated face, a prominent nose, brown eyes, a receding hairline and a short mustache. Although the end results are extremely pixelated, the final renders show great progress in how A.I. can generate faces from scratch.

Karnewar says he plans to scale out the project to integrate additional datasets such as Flicker8K and Coco captions. Eventually, T2F could be used in the law enforcement field to identify victims and/or criminals based on text descriptions, among other applications. He’s open to suggestions and contributions to the project.

To access the code and contribute, head to Karnewar’s repository on Github here.

Kevin Parrish
Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…
I found the best Prime Day deal on a tablet hidden beyond Amazon
Microsoft Surface Pro 12-inch, stylus, and keyboard.

A good tablet can take your productivity to the next level, but a boring one will find a niche use and eat dust on a table or couch for most of its time. I love iPads and have been pushing them – as far as I can — to act as my primary computing machine for nearly half a decade now. It has never managed to replace a proper laptop, like a MacBook Air or a Windows machine. 

Why not buy a Windows laptop, you might ask? Well, Windows-powered tablets, especially those Surface devices sold by Microsoft, are pretty expensive. I love the new 12-inch Surface Pro, but at $799, it felt like a steep purchase despite its impressive specifications. 

Read more
Prime Day is over, but this powerful Dell laptop is still at its lowest price
The Dell Vostro 3530 laptop on a white background.

Prime Day is already over, but that doesn't mean that there are no more laptop deals for you to shop on Amazon. Here's one that caught our eye -- the Dell Vostro 3530 with 32GB of RAM for its lowest-ever price of $649, following a 28% discount on its original price of $899. This limited-time offer of $250 off may not last much longer though, so if you want to take advantage of this bargain, we highly recommend that you finalize your purchase for this device as soon as you can.

Buy Now

Read more
This Lenovo all-in-one computer is 30% off even though Prime Day has ended
The Lenovo V100 All-in-One Desktop Computer on a white background.

Even though Prime Day is already finished, there are still some excellent desktop computer deals on Amazon. If you don't have much space, or you just want to avoid clutter, you should take a look at the Lenovo V100 All-in-One PC. From its original price of $800, it's all the way down to just $560 following a 30% discount. The offer for this machine will only be available for a limited time though, so you need to act fast and proceed with the transaction immediately to secure the savings of $240.

Buy Now

Read more