Huawei’s A.I. has finished Schubert’s Unfinished Symphony, and we’ve heard it

Schubert left Symphony No. 8 unfinished. A smartphone’s A.I. just completed it

Andy Boxall/Digital Trends

Franz Schubert composed his Symphony No.8 in 1822, but never completed it, making only two movements along with an outline of a third. Nearly 200 years later, Huawei, Emmy-awarding composer Lucas Cantor, and artificial intelligence (A.I.) inside the Mate 20 Pro smartphone have done what the renowned composer didn’t. They’ve finished the unfinished symphony.

The project is a continued illustration of not only the power of Huawei’s Kirin 980 processor and Dual-Neural Processing Unit (NPU) artificial intelligence accelerator, but also the potential for using A.I. in varied creative projects. We’re familiar with A.I. modes on smartphone cameras, and Huawei has previously demonstrated the power and speed of its A.I. in a self-driving car, where a phone was used to identify and help avoid obstacles.

Surely, completing Schubert’s No. 8 Symphony is an entirely different proposition? Can a smartphone and on-board A.I. really finish one of the most famous unfinished musical works, and do so in a way that we’d want to hear? Digital Trends attended the first ever public performance of the completed piece in London, and spoke to Lucas Cantor and Huawei about this fascinating project.

The technology

Just how much did the Huawei Mate 20 Pro and its A.I. contribute?

“The entire melody [for the third and fourth movements] was composed using artificial intelligence and the Mate 20 Pro,” Huawei’s head of handset portfolio, Arne Herkelmann, told Digital Trends.

Andy Boxall/Digital Trends

The phone and its chipset are entirely unmodified, and exactly the same as the Mate 20 Pro you may have in your pocket. But Huawei made a specific A.I. model to understand and develop the music, which was given Schubert’s first and second movements, and then by analyzing the timbre, pitch, and meter of the music, it generated the third and fourth movements.

Training the A.I. was almost like sending it to school, just with a laser focus on a single subject — make music like Schubert. MIDI files were used, rather than school books, from which it learned patterns and structure. Huawei generated 20 different models over the six month project, before settling on a final one, during which time the A.I. was given around 90 pieces of Schubert’s work as reference.

“I think I responded ‘yes’ before I even finished reading the email.”

“In the beginning it was all Schubert, so the A.I. could understand what Schubert did,” Herkelmann said. “In the later process, we also trained it on composers we knew influenced Schubert to make sure there was this richer background and context to the work.”

This approach helped the A.I. better understand its task. It also needed training to make sure it came up with a melody suitable for Schubert, and didn’t suddenly make a melody better suited to a bizarre pop version of Schubert’s unfinished symphony.

As the melody came to life, it was composer Lucas Cantor’s job to orchestrate the piece, to give it emotion, and to help teach the A.I. and correct its mistakes. It took him just 30 days to come up with the completed third and fourth movements of the 8th Symphony. He called this an otherwise impossible time-frame without the A.I.

The composer

Lucas Cantor is an Emmy-awarding winning composer currently writing music for DreamWorks Animation, among other projects, and he turned out to be the ideal man for the job. Long before the project, his personal interest in artificial intelligence was the basis for a friendship with one of the engineers working on Huawei’s A.I., a fortuitous meeting that eventually led to the company contacting him to finish the unfinished symphony using A.I.

Arne Herkelmann (left), Huawei’s head of handset portfolio, and Lucas Cantor, an Emmy-awarding composer. Andy Boxall/Digital Trends

“I think I responded ‘yes’ before I even finished reading the email,” Cantor laughed, describing the way he became involved. “I was excited by the potential.”

Although the idea of using artificial intelligence and technology to create, enhance, or augment music sounds futuristic, it has been going on in one form or another for decades.

“The technology I use to express myself musically is all digital,” Cantor said, “and I couldn’t express myself so quickly without it. I think if Schubert had it, he would have been 10 times more prolific.”

“What you’re hearing is a coming together of many years of expertise.”

It’s easy to dismiss A.I., or perhaps even Cantor, as a sleeping partner — that one was more influential than the other. So what was the split? How much of the completed piece is Cantor, and how much is Huawei’s A.I.?

“It’s a collaboration, like any piece of music,” he said. “I’ve collaborated with humans in much the same way as I did the A.I.; the A.I. generated melodies, primarily what it thought would be indicative of Schubert, or what Schubert may have come up with.”

The emotion

What Cantor brings is understanding, experience, and emotion — things the A.I. is unable to replicate.

“What you’re hearing is a coming together of many years of expertise,” he said, speaking about himself, the conductor, the musicians, and Huawei’s technological knowledge. One could not have done it without the other.

Andy Boxall/Digital Trends

“The smartphone couldn’t have written it without me, but it would not have been the same piece without the smartphone,” he said, saying it was like working with the “ultimate brainstorming partner.” “I didn’t start writing until I saw what the A.I. had done. I had some ideas of how the unfinished symphony would go, a framework of the form. The first time I heard the melodies made by the smartphone, I was struck by how interesting they were. There’s this one melody in the third movement that that has a major third going to a minor third that makes it really eerie, but very beautiful. It’s not a melody I would have come up with.”

The deeper we spoke about the way the movements had been created, the more Cantor revealed about the A.I.’s ability.

If I wasn’t told the third and fourth movements were created by artificial intelligence, I wouldn’t have known.

“The emotional depth in the last two movements is something that I brought, but in a direction the smartphone and I charted together. I was struck by the smartphone generating melodies not appropriate for 1822, but still seemed to fit the piece really well. That was a leap that the smartphone made, and I followed it. I thought it was a great idea.”

Fascinatingly, Cantor talks about beginning to form a relationship with the smartphone and the A.I. At times through our conversation, he spoke about it in a manner with which one may talk about another person.

“I know it doesn’t have feelings, but I keep ascribing feelings to it,” he said, laughing.

Andy Boxall/Digital Trends

As unusual as it sounds, doing this may have had an interesting effect on the finished piece.

“Me attributing feelings to it helped me get a more robust creative product out of it,” Cantor said.

The finished symphony

Performed at London’s Cadogan Hall in prestigious Chelsea by the 67-piece English Session Orchestra, the completed piece lasted for 48 minutes, compared to the 27 minutes of the original two movements. If I wasn’t told the third and fourth movements were created by artificial intelligence, I wouldn’t have known. There was nothing digital, generic, or soulless about it.

Playful, soaring, and at times rather epic, Cantor’s A.I.-generated movements carried over motifs from Schubert’s original movements, and managed to add unique elements to them. Not being a music critic, or classical music expert, it’s difficult to say how well the new movements “fit” with the originals, but that’s not really the point.

Andy Boxall/Digital Trends

None of the team set out to create the definitive finished version of the unfinished symphony, and it’s not the first time the piece has been completed by other musicians either.

The answer of whether it was a success, or not, will be a subjective one. However, we can say that a memorable, emotional piece of music was created. We shouldn’t really be surprised that it worked so well either. From the electronic music of Jean Michel Jarre, bands like the Pet Shop Boys being creatively enabled by the Atari ST computer, and digital vocalists like Hatsune Miku and Crypton Future Media’s other Vocaloid software giving musicians the chance to make music they wouldn’t normally be able to — music and high-tech have been converging in fascinating new ways for years.

The future

What will A.I. finishing the Unfinished Symphony mean? Is it purely a promotional tool for Huawei, showcasing its A.I. talent, or is there something more? While we doubt Huawei will start finishing the many other unfinished pieces out there, the music industry is already paying attention to the possibilities. Singer-songwriter Taryn Southern used A.I. programs to produce her I Am AI album, for example.

Andy Boxall/Digital Trends

The potential for artificial intelligence to be used in creative scenarios is being seen across all industries, not just music. Paintings made by A.I. have sold for hundreds of thousands of dollars, A.I. in our smartphones is helping us become better photographers, and A.I. in editing tools is helping professionals and amateurs alike make images look even better. Artificial intelligence is already a creative force.

We asked Cantor if this was a one-time thing for him.

“I certainly hope it’s not the last time [I collaborate with A.I.], it has been fantastic,” he said.

But what about the wider impact of music and artificial intelligence coming together? He made an exciting prediction.

“In a few years, I don’t think it will be unusual at all to compose music this way,” Cantor said.

Mobile

It’s 2025. How has 5G changed our lives? We asked experts to predict the future

2025 is the year that mobile carriers say 5G goes mainstream. How will the technology change, revolutionize, or make obsolete the things we do today? We asked futurists to give us an idea of our future 5G connected world.
Emerging Tech

Amazon’s facial recognition updates can detect fear, among other emotions

Amazon’s facial recognition software can detect emotion on people’s faces. The company announced improvements in emotion detection, including: Happy, sad, angry, surprised, disgusted, calm, confused, and fear.
Cars

Chevrolet’s one-of-a-kind electric Camaro drag racer is now for sale

The Chevrolet eCOPO Camaro concept is an all-electric drag racer that points toward a future of zero-emission performance. The 700-horsepower eCOPO can run the quarter mile in under 10 seconds, according to Chevy.
Mobile

Is Sony’s smartphone business dead or can it be revived?

After its worst quarter ever, where Sony Mobile shipped less than 1 million smartphones for the first time, the future looks decidedly bleak. Can Sony’s smartphone business be turned around, or is it time to call it?
Mobile

Go monochrome with some of the best black-and-white photo apps

Despite the ubiquity of color images native to digital cameras and smartphones, the emotion and drama of black-and-white photography maintain their allure. Here are a few of our favorite monochrome shooting and editing apps.
Mobile

Motorola One Action says goodbye to annoying vertical videos

The new midrange Motorola One Action features an ultra-wide-angle lens that means you can take horizontal video while filming vertically. Here's everything you need to know about the Motorola One Action.
Mobile

Critical Bluetooth security bug discovered. Protect yourself with a quick update

Researchers have discovered a major new security flaw in Bluetooth, which could leave millions of devices at risk of a malicious hack. The attack allows a hacker to “break” Bluetooth security without anyone knowing.
Computing

A dead pixel doesn't mean a dead display. Here's how to repair it

Dead pixel got you down? We don't blame you. Check out our guide on how to fix a dead pixel and save yourself that costly screen replacement or an unwanted trip to your local repair shop.
Deals

Score this Amazon-renewed Apple iPhone XS at a $350 discount

iPhones are pretty expensive pieces of technology, but if you don’t mind buying a renewed model, you can save quite a bit of money. Amazon is offering renewed unlocked versions of the Apple iPhone XS at a $350 discount.
Deals

Best smartwatch deals for August 2019: Samsung, Fitbit, and Apple Watch sales

Smartwatches make life easier by sending alerts right on your wrist. Many also provide fitness-tracking features. If you're ready to take the plunge into wearables and want to save money, here are the best smartwatch deals for August 2019.
Mobile

Learn how to make your iPhone play local radio in iOS 13

Find out exactly how to use the Live Radio in iOS 13: Our guide will explain how iOS 13 connects to live radio, what commands to give, and how to revisit past stations. Enjoy your favorite live radio right from your iPhone.
Deals

Looking for a good cheap phone? Get the Samsung Galaxy A10 for $145 on Amazon

Samsung's A-Series lineup features phone models from the low-end to the mid-tier. One of its cheaper ones, the Galaxy A10, is perhaps watered-down specs-wise but still boasts enough workable features that make it recommendable.
Mobile

These fraudulent Android apps were downloaded 8 million times

According to a new report from security research firm Trend Micro, a hefty 85 Android apps have been caught serving fraudulent ads that take over the user's screen -- and those apps have been downloaded 8 million times.
Deals

Score this Samsung Galaxy S9 Plus with a huge $101 discount at Amazon

If you're thinking about upgrading your phone and not willing to expend over $800 on the S10, the Samsung Galaxy S9 Plus is a great option. It's discounted by $101 at Amazon, dropping its price from $700 to only $599.