Marimba-playing robot uses deep-learning AI to compose and perform its own music

When the inevitable robot invasion happens, we now know what the accompanying soundtrack will be — and we have to admit that it’s way less epic than the Terminator 2: Judgment Day theme. Unless you’re a massive fan of the marimba, that is!

That assertion is based on research coming out of the Georgia Institute of Technology, where engineers have developed a marimba-playing robot with four arms and eight sticks that is able to write and perform its own musical compositions. To do this, it uses a dataset of 5,000 pieces of music, combined with the latest in deep learning neural network-based AI.

“This is the first example of a robot composing its own music using deep neural networks,” Ph.D. student Mason Bretan, who first began working on the so-called Shimon robot seven years ago, told Digital Trends. “Unlike some of the other recent advances in autonomous music generation from research being done in academia and places like Google, which is all simulation done in software, there is an extra layer of complexity when a robotic system that lives in real physical three-dimensional space generates music. It not only needs to understand music in general, but also to understand characteristics about its embodiment and how to bring its musical ‘ideas’ to fruition.”

Training Shimon to generate new pieces of music involves first coming up with a numerical representation of small chunks of music, such as a few beats or a single measure, and then learning how to sequence these chunks. Two separate neural networks are used for the work — with one being an “autoencoder” that comes up with a concise numerical representation, and the second being a long short-term memory (LSTM) network that models sequences from these chunks.

“These sequences come from what is seen in human compositions such as a Chopin concerto or Beatles’ piece,” Bretan continued. “The LSTM is tasked with predicting forward, which means given the first eight musical chunks, it must predict the ninth. If it is able to successfully to do this, then we can provide the LSTM a starting seed and let it continue to predict and generate from there. When Shimon generates, it makes decisions that are not only based off this musical model, but also include information about its physical self so that its musical decisions are optimized for its specific physical constraints.”

It’s pretty fascinating stuff. And while the idea of a music-generating bot might sound of interest only to people studying music, the bigger questions it raises about computational creativity are only going to get more important as time goes on.

“Though we are focusing on music, the more general questions and applications pertain to understanding the processes of human creativity and decision-making,” Bretan said. “If we are able to replicate these processes, then we are getting closer to having a robot successfully survive in the real world, in which creative decision-making is a must when encountering new scenarios and problems each day.”

Emerging Tech

Ancient crater the size of NYC discovered under the Greenland ice sheet

A huge crater has been discovered beneath the ice of Greenland, and is thought to be the result of a meteorite impact millions of years ago. The crater is one of the largest ever discovered, measuring 19 miles across.
Movies & TV

The best movies on Amazon Prime right now (November 2018)

Prime Video provides subscribers with access to a host of fantastic films, but sorting through the catalog can be an undertaking. Luckily, we've done the work for you. Here are the best movies on Amazon Prime Video right now.
Home Theater

The best movies on Netflix in November, from 'Buster Scruggs’ to ‘Dracula’

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, witty humor, or anything else.
Movies & TV

The best shows on Netflix, from 'The Haunting of Hill House’ to ‘The Good Place’

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.
Gaming

These are the best Xbox One games available right now

More than four years into its life span, Microsoft's latest console is finally coming into its own. From 'Cuphead' to 'Halo 5,' the best Xbox One games offer something for everyone.
Emerging Tech

Here’s how the InSight mission to Mars will confirm its landing to NASA

NASA's InSight mission has sent a lander to Mars. NASA researchers have now shared details on how they will monitor the touching down of the lander at the end of its 91 million mile journey.
Emerging Tech

Would you swap your keycard for a microchip implant? For many, the answer is yes

Put down your keycard! More people are turning to implanted RFID chips as their choice of workplace identification. Should we be worried about a world in which employees get microchipped?
Outdoors

‘Super magnesium’ may be the next wonder material for outdoor gear

Super Magnesium is a wonder material that is 30 percent lighter than aluminum, as strong as carbon fiber, cheaper to make, and 100-percent recyclable, making it much better for the environment.
Emerging Tech

Forget joysticks — the Guts Game is controlled by a sensor that you swallow

Researchers have created an unusual new game in which players swallow a biosensor and then compete to raise or lower the temperature in their gut. Sound crazy? Here's why it could catch on.
Emerging Tech

Step inside the Nepalese restaurant staffed by robot waiters

A robotics startup from Nepal has created a robot waiter called Ginger. It's capable of delivering food from kitchen to table, and can even engage customers in a bit of friendly banter as it does so.
Emerging Tech

Doctors could soon ditch stitches and seal skin wounds with lasers

Just like the dermal regenerator in Star Trek, physicians may soon be able to heal skin wounds using smart, laser-based technology. That's thanks to researchers from Arizona State University.
Emerging Tech

From tornado flushes to remote controls, modern toilets are flush with tech

With the global observance of World Toilet Day on November 19, we take a look at how the modern toilet in our homes and businesses have evolved, and how they are becoming smarter tools in the future.
Emerging Tech

NASA selects the all-important landing site for its Mars 2020 rover mission

NASA said on Monday that the landing site for its much-anticipated Mars 2020 rover mission has the potential to "revolutionize how we think about Mars and its ability to harbor life."
Emerging Tech

NASA’s ‘space wheat’ is helping earthbound farmers grow crops quicker

Could NASA technology for growing plants on other planets help farmers improve crop yield here on Earth? According to researchers in Australia and the U.K., the answer is a resounding yes.