Skip to main content

Neural network can create high-res images based on a text description

Image Synthesis From Text With Deep Learning | Two Minute Papers
As far as artificial intelligence goes, 2016 has been the year of deep learning. Brain-inspired neural networks have received massive amounts of investment in time, resources and funding — and, boy, has it ever paid off!

In a new piece of research — carried out by investigators at Rutgers University, the University of North Carolina at Charlotte, Lehigh University, and the Chinese University of Hong Kong — neural networks have been used to generate high quality images based on nothing more detailed than basic text descriptions.

Related Videos

“Generating realistic images from text descriptions has many applications,” researcher Han Zhang told Digital Trends. “Previous approaches have difficulty in generating high resolution images, and their synthesized images in many cases lack details and vivid object parts. Our StackGAN for the first time generates 256 x 256 images with photo-realistic details.”

A video of the work was shared online by YouTuber Károly Zsolnai-Fehér as part of his excellent series of Two Minute Papers educational videos.

“For many years, we have trained neural networks to perform tasks like face, traffic sign, or handwriting recognition,” Zsolnai-Fehér told us. “Generally, with millions of training examples, we show the neural network how to do something, and expect them to learn these concepts, and do well on their own afterwards. This piece of work is completely different: here, after learning the neural networks are able to create something completely new — such as synthesizing new, photorealistic images from a piece of text we have written. This opens up a world of possibilities, and I am super-excited to see where researchers take this concept in the future.”

While there have certainly been examples of computational creativity before — ranging from MIT’s Nightmare Machine to projects that can generate predictive video simply by looking at a still image — this is nonetheless an intriguing piece of work. It’s also fascinating because the two-stage method of drawing images looks, to our way of thinking, a whole lot like the way artists will sketch out a piece of work, and then do a second pass to add detail.

We may still be a way from replacing human illustrators with robots, but this is nonetheless an exciting leap forward.

Editors' Recommendations

Here’s how ChatGPT could solve its major plagiarism problem
Close up of ChatGPT and OpenAI logo.

ChatGPT is a wonderful tool but there's a dark side to this advanced AI service that can write like an expert on almost any topic -- plagiarism. When students that are supposed to be demonstrating their knowledge and understanding of a topic cheat by secretly using ChatGPT, it invalidates testing and grading. AI skills are great but aren't the only subject that students should learn.

Policing this problem has proven to be difficult. Since ChatGPT has been trained on a vast dataset of human writing, it's nearly impossible for an instructor to identify whether an essay was created by a student or a machine. Several tools have been created that attempt to recognize AI-generated writing, but the accuracy was too low to be useful.

Read more
This AI can spoof your voice after just three seconds
man speaking into phone

Artificial intelligence (AI) is having a moment right now, and the wind continues to blow in its sails with the news that Microsoft is working on an AI that can imitate anyone’s voice after being fed a short three-second sample.

The new tool, dubbed VALL-E, has been trained on roughly 60,000 hours of voice data in the English language, which Microsoft says is “hundreds of times larger than existing systems”. Using that knowledge, its creators claim it only needs a small smattering of vocal input to understand how to replicate a user’s voice.

Read more
The best AI image generators to create art from text
Théâtre D’opéra Spatial AI artwork developed by Jason Allen.

AI image generators are becoming a hot topic online, but they are far from new. The technology for these tools has been around for some time. It is just reaching a point where they are more accessible to the everyday user.

Some of these text-to-art generators are free, while some are behind paywalls, and others allow for a trial. There are also many styles of art you can create from different generators. Take a look at some of the best AI image generators to see which ones might match your artistic style.
What is an AI image generator?
An AI image generator is essentially a tool that uses machine learning to create art. In its simplest form, it will use text prompts to describe the type of art you want to create, and then it'll do its best job to make it for you. Some tools include additional styles and parameters to their generators to make the results more unique.

Read more