Huawei’s A.I. has finished Schubert’s Unfinished Symphony, and we’ve heard it

Schubert left Symphony No. 8 unfinished. A smartphone’s A.I. just completed it

Andy Boxall/Digital Trends

Franz Schubert composed his Symphony No.8 in 1822, but never completed it, making only two movements along with an outline of a third. Nearly 200 years later, Huawei, Emmy-awarding composer Lucas Cantor, and artificial intelligence (A.I.) inside the Mate 20 Pro smartphone have done what the renowned composer didn’t. They’ve finished the unfinished symphony.

The project is a continued illustration of not only the power of Huawei’s Kirin 980 processor and Dual-Neural Processing Unit (NPU) artificial intelligence accelerator, but also the potential for using A.I. in varied creative projects. We’re familiar with A.I. modes on smartphone cameras, and Huawei has previously demonstrated the power and speed of its A.I. in a self-driving car, where a phone was used to identify and help avoid obstacles.

Surely, completing Schubert’s No. 8 Symphony is an entirely different proposition? Can a smartphone and on-board A.I. really finish one of the most famous unfinished musical works, and do so in a way that we’d want to hear? Digital Trends attended the first ever public performance of the completed piece in London, and spoke to Lucas Cantor and Huawei about this fascinating project.

The technology

Just how much did the Huawei Mate 20 Pro and its A.I. contribute?

“The entire melody [for the third and fourth movements] was composed using artificial intelligence and the Mate 20 Pro,” Huawei’s head of handset portfolio, Arne Herkelmann, told Digital Trends.

Andy Boxall/Digital Trends

The phone and its chipset are entirely unmodified, and exactly the same as the Mate 20 Pro you may have in your pocket. But Huawei made a specific A.I. model to understand and develop the music, which was given Schubert’s first and second movements, and then by analyzing the timbre, pitch, and meter of the music, it generated the third and fourth movements.

Training the A.I. was almost like sending it to school, just with a laser focus on a single subject — make music like Schubert. MIDI files were used, rather than school books, from which it learned patterns and structure. Huawei generated 20 different models over the six month project, before settling on a final one, during which time the A.I. was given around 90 pieces of Schubert’s work as reference.

“I think I responded ‘yes’ before I even finished reading the email.”

“In the beginning it was all Schubert, so the A.I. could understand what Schubert did,” Herkelmann said. “In the later process, we also trained it on composers we knew influenced Schubert to make sure there was this richer background and context to the work.”

This approach helped the A.I. better understand its task. It also needed training to make sure it came up with a melody suitable for Schubert, and didn’t suddenly make a melody better suited to a bizarre pop version of Schubert’s unfinished symphony.

As the melody came to life, it was composer Lucas Cantor’s job to orchestrate the piece, to give it emotion, and to help teach the A.I. and correct its mistakes. It took him just 30 days to come up with the completed third and fourth movements of the 8th Symphony. He called this an otherwise impossible time-frame without the A.I.

The composer

Lucas Cantor is an Emmy-awarding winning composer currently writing music for DreamWorks Animation, among other projects, and he turned out to be the ideal man for the job. Long before the project, his personal interest in artificial intelligence was the basis for a friendship with one of the engineers working on Huawei’s A.I., a fortuitous meeting that eventually led to the company contacting him to finish the unfinished symphony using A.I.

Arne Herkelmann (left), Huawei’s head of handset portfolio, and Lucas Cantor, an Emmy-awarding composer. Andy Boxall/Digital Trends

“I think I responded ‘yes’ before I even finished reading the email,” Cantor laughed, describing the way he became involved. “I was excited by the potential.”

Although the idea of using artificial intelligence and technology to create, enhance, or augment music sounds futuristic, it has been going on in one form or another for decades.

“The technology I use to express myself musically is all digital,” Cantor said, “and I couldn’t express myself so quickly without it. I think if Schubert had it, he would have been 10 times more prolific.”

“What you’re hearing is a coming together of many years of expertise.”

It’s easy to dismiss A.I., or perhaps even Cantor, as a sleeping partner — that one was more influential than the other. So what was the split? How much of the completed piece is Cantor, and how much is Huawei’s A.I.?

“It’s a collaboration, like any piece of music,” he said. “I’ve collaborated with humans in much the same way as I did the A.I.; the A.I. generated melodies, primarily what it thought would be indicative of Schubert, or what Schubert may have come up with.”

The emotion

What Cantor brings is understanding, experience, and emotion — things the A.I. is unable to replicate.

“What you’re hearing is a coming together of many years of expertise,” he said, speaking about himself, the conductor, the musicians, and Huawei’s technological knowledge. One could not have done it without the other.

Andy Boxall/Digital Trends

“The smartphone couldn’t have written it without me, but it would not have been the same piece without the smartphone,” he said, saying it was like working with the “ultimate brainstorming partner.” “I didn’t start writing until I saw what the A.I. had done. I had some ideas of how the unfinished symphony would go, a framework of the form. The first time I heard the melodies made by the smartphone, I was struck by how interesting they were. There’s this one melody in the third movement that that has a major third going to a minor third that makes it really eerie, but very beautiful. It’s not a melody I would have come up with.”

The deeper we spoke about the way the movements had been created, the more Cantor revealed about the A.I.’s ability.

If I wasn’t told the third and fourth movements were created by artificial intelligence, I wouldn’t have known.

“The emotional depth in the last two movements is something that I brought, but in a direction the smartphone and I charted together. I was struck by the smartphone generating melodies not appropriate for 1822, but still seemed to fit the piece really well. That was a leap that the smartphone made, and I followed it. I thought it was a great idea.”

Fascinatingly, Cantor talks about beginning to form a relationship with the smartphone and the A.I. At times through our conversation, he spoke about it in a manner with which one may talk about another person.

“I know it doesn’t have feelings, but I keep ascribing feelings to it,” he said, laughing.

Andy Boxall/Digital Trends

As unusual as it sounds, doing this may have had an interesting effect on the finished piece.

“Me attributing feelings to it helped me get a more robust creative product out of it,” Cantor said.

The finished symphony

Performed at London’s Cadogan Hall in prestigious Chelsea by the 67-piece English Session Orchestra, the completed piece lasted for 48 minutes, compared to the 27 minutes of the original two movements. If I wasn’t told the third and fourth movements were created by artificial intelligence, I wouldn’t have known. There was nothing digital, generic, or soulless about it.

Playful, soaring, and at times rather epic, Cantor’s A.I.-generated movements carried over motifs from Schubert’s original movements, and managed to add unique elements to them. Not being a music critic, or classical music expert, it’s difficult to say how well the new movements “fit” with the originals, but that’s not really the point.

Andy Boxall/Digital Trends

None of the team set out to create the definitive finished version of the unfinished symphony, and it’s not the first time the piece has been completed by other musicians either.

The answer of whether it was a success, or not, will be a subjective one. However, we can say that a memorable, emotional piece of music was created. We shouldn’t really be surprised that it worked so well either. From the electronic music of Jean Michel Jarre, bands like the Pet Shop Boys being creatively enabled by the Atari ST computer, and digital vocalists like Hatsune Miku and Crypton Future Media’s other Vocaloid software giving musicians the chance to make music they wouldn’t normally be able to — music and high-tech have been converging in fascinating new ways for years.

The future

What will A.I. finishing the Unfinished Symphony mean? Is it purely a promotional tool for Huawei, showcasing its A.I. talent, or is there something more? While we doubt Huawei will start finishing the many other unfinished pieces out there, the music industry is already paying attention to the possibilities. Singer-songwriter Taryn Southern used A.I. programs to produce her I Am AI album, for example.

Andy Boxall/Digital Trends

The potential for artificial intelligence to be used in creative scenarios is being seen across all industries, not just music. Paintings made by A.I. have sold for hundreds of thousands of dollars, A.I. in our smartphones is helping us become better photographers, and A.I. in editing tools is helping professionals and amateurs alike make images look even better. Artificial intelligence is already a creative force.

We asked Cantor if this was a one-time thing for him.

“I certainly hope it’s not the last time [I collaborate with A.I.], it has been fantastic,” he said.

But what about the wider impact of music and artificial intelligence coming together? He made an exciting prediction.

“In a few years, I don’t think it will be unusual at all to compose music this way,” Cantor said.

Gaming

Bethesda's sharp new Elder Scrolls mobile game is blunted by microtransactions

Elder Scroll: Blades brings elements of the most popular open-world RPG to mobile phones. It has many of the features fans will expect but, in its current Early Access state, lacks the real charm and depth of its predecessors.
Smart Home

Virtually walk through dream homes with Zillow’s new A.I.-powered 3D home tours

Home sellers and real estate agents listing properties on Zillow's home marketplace now can add 3D tours for free to their listings. Zillow 3D Home uses artificial intelligence to create tours with 360-degree panoramic photos.
Home Theater

Amazon’s free Spotify competitor is here. Just ask Alexa

Just ask Alexa to play your favorite song. Amazon has launched a free, ad-based music streaming service to compete with Spotify's free tier on its popular Echo devices, aiming to bolster subscriptions to Amazon Music Unlimited.
Home Theater

What’s new on Amazon Prime Video and what’s leaving in May 2019

Amazon Prime Video adds new titles each month that are available for free to all Prime members. Check out our list to find all the content hitting Amazon Prime Video in April and May, from new original series to classic films.
Mobile

BlackBerry Messenger to shut down in May, be replaced by enterprise version

BlackBerry Messenger for consumers will shut down at the end of May, nearly six years after it was launched. The app is going back to its roots, in a way, as it is being replaced by the much simpler BlackBerry Messenger Enterprise.
Deals

Apple Watch Series 3 and Samsung Gear S3 smartwatches get big price cuts

The Apple Watch Series 3 and Samsung Gear S3 are at some of the lowest prices we've seen since so far this year. With discounts from Amazon and Walmart, now is a great time to find spring savings on wearables.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Robots that eat landmines and clean your floors

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it's fun to gawk!
Mobile

These are the key settings to change on the LG G8 ThinQ smartphone

The LG G8 ThinQ is finally available. There are a lot of settings turned off by default that may be useful in improving the experience of using the phone. We've rounded up 11 key settings for you to change on the G8.
Mobile

LG reportedly files patent for triple selfie camera on smartphone

LG has reportedly filed a patent for a smartphone with three front-facing cameras. However, if the patent turns out to be true, it may come down as just another gimmick for the struggling mobile phone manufacturer.
Mobile

Best iPhone 7 Plus screen protectors to shield your big, beautiful display

Cracked screens are expensive to replace. Fortunately for you, we've rounded up what's available in terms of protection for Apple's large iPhone 7 Plus. Here are the best screen protectors you can buy.
Mobile

Samsung says the Galaxy Fold needs more improvements before it's released

The Samsung Galaxy Fold has arrived, and it goes on sale soon. Folding out from a 4.6-inch display to a tablet-sized 7.3-inch display, this unique device has six cameras, two batteries, and special software to help you use multiple apps.
Mobile

Add style and functionality with the best iPad Pro 9.7 cases

No tablet should be without protection, especially an expensive one. Check out our picks for the best iPad Pro 9.7 cases, so you can add some protection, functionality, and style to your new device.
Mobile

Amazon cuts prices on Apple Watch Series 4 and Series 3 smartwatches

In the market for a new smartwatch? The Apple Watch is the best one, and right now it's a little more affordable than it usually is. The Apple Watch Series 4 and Apple Watch Series 3 are both available at discounted prices straight from…
Home Theater

Bose QuietComfort 35 II wireless headphones get a $93 price cut

Good head-fi is expensive, but a premium set of headphones is worth it. If you're in for a pair, then you're in luck: The excellent Bose Quietcomfort 35 Series II is on sale right now from Rakuten with a two-day coupon code that can save…