Skip to main content

Can machines be creative? Meet the Google coders teaching them to make art

Machine learning & art - Google I/O 2016
Can a machine be creative? Google thinks so, and it has an entire team dedicated to teaching machines how to view the world a little more like us emotional humans.

Think about computers as if they were children and it’s simple to understand how coders can teach them to learn. Artificial intelligence is, at the beginning, very basic and simple. Human moderators instruct computers, showing them how to think and thus teach themselves. Once the coders give them the basics, though, they can expand that knowledge quickly.

“What can you do with 7 million digital artifacts?”

At the Google Cultural Institute in Paris, France, the search giant is teaching machines how to categorize 7 million images of human artistic achievement throughout the centuries. The Institute even has a website, as well as apps for iOS and Android where you can search through works of art from different museums around the globe. To create its catalog of art, the code artists in residence at the Institute had to teach computers to view images the way humans would to create an accurate digital archive of art throughout human history.

Cataloging history is well and good, but some of the skills computers are learning from sorting and filing are actually making them more creative. The artists in residence are now experimenting with computers to create new works of art using machine intelligence and the catalog of 7 million images they’ve pieced together. During Google I/O 2016,  Cyril Diagne and Mario Klingemann explained how they’ve taught machines to see art like humans, and how they’ve trained machines to be creative.

Teaching computers their ABCs

One of the first things you teach a child is language. In Western culture, that means learning your ABCs. Mario Klingemann, a self-described code artist from Germany, started teaching machines to identify stylized letters from old texts to find out if he could teach a computer to recognize thousands of different-looking As, Bs, Cs, and so on. It was a crash course in teaching machines how to categorize images the way humans would.

While a computer may look at a stylized letter B covered in vines and flowers and see a plant of some kind, even a 5-year-old child could immediately identify the image as a letter B — not a plant. To teach his computer to recognize its ABCs, Klingemann fed it thousands of images of stylized letters. He created a Tinder-like interface of swiping right or left to tell his machines if they guessed the letter right or wrong.

Letters machine
Image used with permission by copyright holder

It turns out, machines do learn their ABCs pretty quickly; they started seeing letters in everything. Just as humans see faces in clouds and images in abstract artwork, his computers saw letters in completely unrelated images. Klingemann showed his computer a drawing or etching of a ruined building, and they saw a letter B instead.

Klingemann explained that when you train a computer with only one set of images, it starts to see only that kind of image in everything. That’s why his machines saw a letter in a ruin.

Teaching computers to categorize 7 million images

When Digital Interaction Artist Cyril Diagne joined the Cultural Institute, Google posed a rather daunting question to him, “What can you do with 7 million digital artifacts?”

Diagne was overwhelmed by the question, so he charted every image in a gloriously massive sine wave, which you can see below. That wave later ended up becoming a beautiful representation of everything the project hopes to accomplish with machine learning. Diagne’s sine wave is actually searchable, so you can surf a sea of all the images in the digital archive made by the Google Cultural Institute. Images are grouped in categories, and from a bird’s eye view, you just see a sea of dots. As you move in, you can see specific images, all with a common theme, whether it’s puppies, farms, or people.

You can search through it, too, and find the images you want. If you look hard enough, you might even run into what Diagne calls the Shore of Portraits. That’s where all the images of people’s faces are clustered.

To make the searchable map of every image in the archive, Diagne and his team had to create a category for everything to teach the machine what was what.

Categorizing 7 million artifacts, many of which may have multiple categories, is no easy task. The team had to think up some that were outside the box. It’s not enough to just categorize things based on what they are. They also had to create categories for the emotions that images evoke.

Teaching machines human emotions is an important step toward making them more creative.

That way, you can search for an image of “calm,” and the computer will show you images that evoke a sense of calm, like sunsets, serene lakes, and so on. Amazingly, the machines learned how to identify human emotions with such skill that they can put themselves in our shoes to consider how a certain image would make a human feel.

Teaching machines human emotions is an important step toward making them more creative. After all, much of modern art is visual representations of human emotions.

But can a machine be creative?

Creativity and artistry are two things that we humans like to think of as ours alone. Animals don’t make art, nor do machines … yet. Google’s Deep Dream project attempted to turn the notion that machines can’t create art on its head. The search giant trained computers to manipulate images to create bizarre, psychedelic works of art. The images created by Google’s Deep Dream engine may not be pretty, but they certainly are unique and wildly creative. Machine creations contain psychedelic colors, slugs, weird eyes, and disembodied animals swirling in undefined spaces.

Some may argue that it’s not really art if machines are just combining existing images, twisting them, and dipping them in extreme colors; Google would beg to differ, and so would code-artist Klingemann.

“Humans are incapable of original ideas,” he explained.

Even famous paintings contain elements of previous artwork, he noted. Picasso’s 1907 masterpiece Les Demoiselles d’Avignonfor example, has influences from African art and precursors to cubists like Paul Cezanne. For that matter, collages, which combine existing images in an artistic fashion, are another well-established art form. Picasso, Andy Warhol, Man Ray, and more are known for their eccentric collages, so why can’t collages made by machines also stand as art?

Klingemann wanted to push the boundaries of digital art and see how creative machines could get long before he started his residency at the Google Cultural Institute. Using his own less powerful machines, Klingemann started playing around with the Internet Archives and Google’s TensorFlow machine learning software to make digital collages.

He created a machine-learning tool called Ernst, named after the surrealist and collage artist Max Ernst. Klingemann identified a series of objects from Ernst’s work and told his computer to make different collages with the same elements. The results were often surreal, sometimes funny, and at other times, absolutely terrible.

“Humans are incapable of original ideas.”

Klingemann wanted more control over the chaotic images his machines were producing, so he started teaching them new things. He asked himself, “What is interesting to humans?” Klingemann knew he had to train the system what to look for, to teach it how to view all those elements like a human artist would.

The resulting artwork is gorgeous and entirely unique. Although Klingemann obviously used old images to create his work, they’re displayed in a new context, and that makes all the difference.

Right now, computer creativity is limited to interesting collages and understanding which images go well together. Machines aren’t making their own art yet, but the code artists who power them are becoming more curator than creator during the process.

It remains to be seen how far man can expand the creative minds of machines, but it certainly is fascinating to watch.

Editors' Recommendations

Malarie Gokey
Former Digital Trends Contributor
As DT's Mobile Editor, Malarie runs the Mobile and Wearables sections, which cover smartphones, tablets, smartwatches, and…
Google Japan’s bizarre new keyboard can also catch (literal) bugs
A man holds a gboard with a net at the end while a young boy stands next to him

Japan delights us with products like the Pen Pineapple Apple Pen and the Iyashi Octopus Sucker Massager. Google captivates us with joyful products such as Gmail and Pixel phones. So, what happens when you combine Japan with Google? Simple. You get a Gboard bug catcher, of course.

Meet the playful concept for a keyboard from Google Japan. The Gboard stick is a long and thin ruler-like device with alphabetical keys running down its length, to which a net can be attached at the end.

Read more
Google Meet just got a massive advantage over Zoom and Teams
Google Meet's stylized background show movement glitches.

If you've ever tried Google Meet's visual effects, you already know that moving around can create glitches and gaps at the edges where the foreground object (you) meets the stylized background. The computer's confusion about which part of a webcam image belongs in the front and which goes in the back is getting a huge upgrade with the latest Google Meet update.

The issue with the old version is that many computers lack the processing power to extract depth information from a webcam. That means Google Meet might not find the edges well enough to begin with, and it is often slow to update whenever you move. It's not as noticeable when using a blur effect but can spoil the experience when you apply a stylized background. Nothing wrecks a peaceful nature background more than a pixelated hand with pieces missing when you wave.

Read more
Google Meet and Google Duo begin confusing merger
Google services (YouTube, Gmail, Chrome, Duo, Meet, Google Podcasts) icons app on smartphone screen.

Google has begun the next phase of merging its Duo and Meet video chat apps into one service.

The brand sent out an update on Wednesday, which will change the Google Duo icon to Google Meet. The update is largely aesthetic but is set to roll out for Android and iOS devices over the coming weeks. You can expect to receive a home screen notification explaining how Google Duo has been merged into Google Meet.

Read more