Huawei’s A.I. has finished Schubert’s Unfinished Symphony, and we’ve heard it

Schubert left Symphony No. 8 unfinished. A smartphone’s A.I. just completed it

Huawei Unfinished Symphony phone
Andy Boxall/Digital Trends

Franz Schubert composed his Symphony No.8 in 1822, but never completed it, making only two movements along with an outline of a third. Nearly 200 years later, Huawei, Emmy-awarding composer Lucas Cantor, and artificial intelligence (A.I.) inside the Mate 20 Pro smartphone have done what the renowned composer didn’t. They’ve finished the unfinished symphony.

The project is a continued illustration of not only the power of Huawei’s Kirin 980 processor and Dual-Neural Processing Unit (NPU) artificial intelligence accelerator, but also the potential for using A.I. in varied creative projects. We’re familiar with A.I. modes on smartphone cameras, and Huawei has previously demonstrated the power and speed of its A.I. in a self-driving car, where a phone was used to identify and help avoid obstacles.

Surely, completing Schubert’s No. 8 Symphony is an entirely different proposition? Can a smartphone and on-board A.I. really finish one of the most famous unfinished musical works, and do so in a way that we’d want to hear? Digital Trends attended the first ever public performance of the completed piece in London, and spoke to Lucas Cantor and Huawei about this fascinating project.

The technology

Just how much did the Huawei Mate 20 Pro and its A.I. contribute?

“The entire melody [for the third and fourth movements] was composed using artificial intelligence and the Mate 20 Pro,” Huawei’s head of handset portfolio, Arne Herkelmann, told Digital Trends.

huawei ai unfinished symphony phone
Andy Boxall/Digital Trends

The phone and its chipset are entirely unmodified, and exactly the same as the Mate 20 Pro you may have in your pocket. But Huawei made a specific A.I. model to understand and develop the music, which was given Schubert’s first and second movements, and then by analyzing the timbre, pitch, and meter of the music, it generated the third and fourth movements.

Training the A.I. was almost like sending it to school, just with a laser focus on a single subject — make music like Schubert. MIDI files were used, rather than school books, from which it learned patterns and structure. Huawei generated 20 different models over the six month project, before settling on a final one, during which time the A.I. was given around 90 pieces of Schubert’s work as reference.

“I think I responded ‘yes’ before I even finished reading the email.”

“In the beginning it was all Schubert, so the A.I. could understand what Schubert did,” Herkelmann said. “In the later process, we also trained it on composers we knew influenced Schubert to make sure there was this richer background and context to the work.”

This approach helped the A.I. better understand its task. It also needed training to make sure it came up with a melody suitable for Schubert, and didn’t suddenly make a melody better suited to a bizarre pop version of Schubert’s unfinished symphony.

As the melody came to life, it was composer Lucas Cantor’s job to orchestrate the piece, to give it emotion, and to help teach the A.I. and correct its mistakes. It took him just 30 days to come up with the completed third and fourth movements of the 8th Symphony. He called this an otherwise impossible time-frame without the A.I.

The composer

Lucas Cantor is an Emmy-awarding winning composer currently writing music for DreamWorks Animation, among other projects, and he turned out to be the ideal man for the job. Long before the project, his personal interest in artificial intelligence was the basis for a friendship with one of the engineers working on Huawei’s A.I., a fortuitous meeting that eventually led to the company contacting him to finish the unfinished symphony using A.I.

Huawei Unfinished Symphony phone
Arne Herkelmann (left), Huawei’s head of handset portfolio, and Lucas Cantor, an Emmy-awarding composer. Andy Boxall/Digital Trends

“I think I responded ‘yes’ before I even finished reading the email,” Cantor laughed, describing the way he became involved. “I was excited by the potential.”

Although the idea of using artificial intelligence and technology to create, enhance, or augment music sounds futuristic, it has been going on in one form or another for decades.

“The technology I use to express myself musically is all digital,” Cantor said, “and I couldn’t express myself so quickly without it. I think if Schubert had it, he would have been 10 times more prolific.”

“What you’re hearing is a coming together of many years of expertise.”

It’s easy to dismiss A.I., or perhaps even Cantor, as a sleeping partner — that one was more influential than the other. So what was the split? How much of the completed piece is Cantor, and how much is Huawei’s A.I.?

“It’s a collaboration, like any piece of music,” he said. “I’ve collaborated with humans in much the same way as I did the A.I.; the A.I. generated melodies, primarily what it thought would be indicative of Schubert, or what Schubert may have come up with.”

The emotion

What Cantor brings is understanding, experience, and emotion — things the A.I. is unable to replicate.

“What you’re hearing is a coming together of many years of expertise,” he said, speaking about himself, the conductor, the musicians, and Huawei’s technological knowledge. One could not have done it without the other.

Huawei Unfinished Symphony phone
Andy Boxall/Digital Trends

“The smartphone couldn’t have written it without me, but it would not have been the same piece without the smartphone,” he said, saying it was like working with the “ultimate brainstorming partner.” “I didn’t start writing until I saw what the A.I. had done. I had some ideas of how the unfinished symphony would go, a framework of the form. The first time I heard the melodies made by the smartphone, I was struck by how interesting they were. There’s this one melody in the third movement that that has a major third going to a minor third that makes it really eerie, but very beautiful. It’s not a melody I would have come up with.”

The deeper we spoke about the way the movements had been created, the more Cantor revealed about the A.I.’s ability.

If I wasn’t told the third and fourth movements were created by artificial intelligence, I wouldn’t have known.

“The emotional depth in the last two movements is something that I brought, but in a direction the smartphone and I charted together. I was struck by the smartphone generating melodies not appropriate for 1822, but still seemed to fit the piece really well. That was a leap that the smartphone made, and I followed it. I thought it was a great idea.”

Fascinatingly, Cantor talks about beginning to form a relationship with the smartphone and the A.I. At times through our conversation, he spoke about it in a manner with which one may talk about another person.

“I know it doesn’t have feelings, but I keep ascribing feelings to it,” he said, laughing.

Huawei Unfinished Symphony phone
Andy Boxall/Digital Trends

As unusual as it sounds, doing this may have had an interesting effect on the finished piece.

“Me attributing feelings to it helped me get a more robust creative product out of it,” Cantor said.

The finished symphony

Performed at London’s Cadogan Hall in prestigious Chelsea by the 67-piece English Session Orchestra, the completed piece lasted for 48 minutes, compared to the 27 minutes of the original two movements. If I wasn’t told the third and fourth movements were created by artificial intelligence, I wouldn’t have known. There was nothing digital, generic, or soulless about it.

Playful, soaring, and at times rather epic, Cantor’s A.I.-generated movements carried over motifs from Schubert’s original movements, and managed to add unique elements to them. Not being a music critic, or classical music expert, it’s difficult to say how well the new movements “fit” with the originals, but that’s not really the point.

Huawei Unfinished Symphony phone
Andy Boxall/Digital Trends

None of the team set out to create the definitive finished version of the unfinished symphony, and it’s not the first time the piece has been completed by other musicians either.

The answer of whether it was a success, or not, will be a subjective one. However, we can say that a memorable, emotional piece of music was created. We shouldn’t really be surprised that it worked so well either. From the electronic music of Jean Michel Jarre, bands like the Pet Shop Boys being creatively enabled by the Atari ST computer, and digital vocalists like Hatsune Miku and Crypton Future Media’s other Vocaloid software giving musicians the chance to make music they wouldn’t normally be able to — music and high-tech have been converging in fascinating new ways for years.

The future

What will A.I. finishing the Unfinished Symphony mean? Is it purely a promotional tool for Huawei, showcasing its A.I. talent, or is there something more? While we doubt Huawei will start finishing the many other unfinished pieces out there, the music industry is already paying attention to the possibilities. Singer-songwriter Taryn Southern used A.I. programs to produce her I Am AI album, for example.

Huawei Unfinished Symphony phone
Andy Boxall/Digital Trends

The potential for artificial intelligence to be used in creative scenarios is being seen across all industries, not just music. Paintings made by A.I. have sold for hundreds of thousands of dollars, A.I. in our smartphones is helping us become better photographers, and A.I. in editing tools is helping professionals and amateurs alike make images look even better. Artificial intelligence is already a creative force.

We asked Cantor if this was a one-time thing for him.

“I certainly hope it’s not the last time [I collaborate with A.I.], it has been fantastic,” he said.

But what about the wider impact of music and artificial intelligence coming together? He made an exciting prediction.

“In a few years, I don’t think it will be unusual at all to compose music this way,” Cantor said.

Smart Home

Virtually walk through dream homes with Zillow’s new A.I.-powered 3D home tours

Home sellers and real estate agents listing properties on Zillow's home marketplace now can add 3D tours for free to their listings. Zillow 3D Home uses artificial intelligence to create tours with 360-degree panoramic photos.
Movies & TV

Clip from John Wick: Chapter 3 confirms the dog is totally fine

John Wick: Chapter 3 — Parabellum, the third installment of the wildly successful action series that stars Keanu Reeves as a deadly assassin forced out of retirement, hits theaters in May 2019. Here's everything we know about it so far.
Home Theater

Amazon’s free Spotify competitor is here. Just ask Alexa

Just ask Alexa to play your favorite song. Amazon has launched a free, ad-based music streaming service to compete with Spotify's free tier on its popular Echo devices, aiming to bolster subscriptions to Amazon Music Unlimited.
Home Theater

What’s new on Hulu in May 2019, and what’s leaving soon

Our complete list of what's new on Hulu for May 2019, our personal favorites, and which titles will be removed will help you catch up on all the site has to offer -- and ensure you don't miss any titles heading into the streaming ether.
Mobile

Common Samsung Galaxy S10, S10 Plus, and S10e problems and how to fix them

Samsung's new Galaxy S10 range is gorgeous and extremely powerful. But they're not perfect, and you may discover some issues with your new phone. Here are some of the most common Galaxy S10 problems and how to fix them.
Mobile

Michael Kors updates its Sofie smartwatch, but still uses a processor from 2016

Michael Kors announced an update to the Sofie smartwatch, now offering heart rate monitoring, GPS, and NFC support. There's only one problem — the device still offers the Snapdragon Wear 2100 processor.
Product Review

Screen snags aside, the Galaxy Fold is an exciting step toward a foldable future

Samsung's Galaxy Fold is the company's first foldable phone, with two screens, six cameras, and a dual-cell battery. The phone may be delayed due to display issues, but that doesn't stop us from asking -- what's it like to use?
Photography

Capture life in every direction with the best 360 cameras

While 360 cameras are still a new technology, that doesn't mean there's not a few that are worth a look. Whether you want to shoot from the middle or just need a simple, affordable option, here are the best 360 cameras on the market.
Home Theater

The best MP3 players of 2019 cram tons of music into a small package

Want to go for a run, but your phone is weighing you down? Don't sweat it. Can't fit your whole music library on your smartphone? No worries. Check out our list of the best MP3 players, and find one that works for you.
Apple

WWDC 2019 Complete Coverage

Apple’s Worldwide Developer Conference is a key tech event each year, and for Apple fans, it will be one of the two best times of 2019 (along with "new iPhone day," of course). For the last few years, Apple has debuted much of its…
Mobile

Whether by the pool or the sea, make a splash with the best waterproof phones

Whether you're looking for a phone you can use in the bath, or you just want that extra peace of mind, waterproof phones are here and they're amazing. Check out our selection of the best ones you can buy.
Mobile

Walmart drops a killer deal on the Apple iPad Mini 4 tablet

The Apple iPad still reigns supreme in the tablet market, and the pint-sized 7.9-inch iPad Mini is the perfect everyday carry companion. Now’s the perfect chance to score a deal on the last-gen iPad Mini 4 before it’s gone for good.
Mobile

Google Creative Lab’s new AR experiment helps you learn how to draw

Google's Creative Lab developed an experiment called Drawalong AR that could make it easier for aspiring artists to follow YouTube tutorials. The experiment essentially leverages Google's ARCore to create a virtual tracing paper.
Home Theater

Set your ears free with the best completely wireless earbuds

If you can't stand the tangle of cords, or you're just excited about completely wireless earbuds, you're going to need some help separating the wheat from the chaff. Our list serves up the best true wireless earbuds around.