Skip to main content

I met Samsung’s artificial humans, and they showed me the future of A.I.

Andy Boxall/DigitalTrends

This story is part of our continuing coverage of CES 2020, including tech and gadgets from the showroom floor.

What is Neon? Shrouded in mystery leading up to CES 2020, all we knew was that Neon had something to do with artificial intelligence. Was it a Google Assistant competitor? A robot? Something more?

“It’s a preview of a wonderful technology we have, and a wonderful future we can create together,” Neon’s CEO Pranav Mistry said at the start of his keynote presentation.

So what is it? It’s not hyperbole, for a start. Neon is a step closer to living with a digital creation that not only understands and emotes with us in a meaningful and relatable way, but is also able to create valuable memories with us and truly share our lives.

Four months of work

Neon CEO Pranav Mistry Andy Boxall/DigitalTrends

Explaining exactly what Neon is, how it works, and the incredible depth of technology underlying it is a considerable challenge and one that Neon itself isn’t quite sure how to tackle. To help introduce Neon, Mistry started out by saying he wants to change the way we interact with machines, and no longer say just, “Stop,” “Next song,” or even, “Hey Google, Bixby, or Siri,” because it’s not how we talk to humans.

Mistry said he wants to “push the boundaries so machines understand more about us. Whether we are tired or happy, our expressions, and our emotions.”

In turn, the more machines understand us, the more we will be able to connect with them on a deeper, human level. He believes the path to this means machines need to look and act more like us, and this is where Neon’s journey really began.

The CES demonstration came just four months after the project started. Mistry and the team began by creating a digital version of a friend, which closely emulated his facial movements during conversation. This evolved into larger, grander tests until eventually, the digital version began to do things on its own. It would make expressions the real person had not. It had “learned,” and become something individual.

The Neon booth in Central Hall at CES is covered in large screens showing people on them, all moving, smiling, laughing, or silently mouthing words to the audience. Except these aren’t videos. These are Neons. They are digital creations born from real people, and although they visually represent the model on which they’re based, the movements, expressions, and “emotions” are entirely automatically generated.

Once you understood this, it was surreal walking around the booth, looking at the Neons who in turn are looking at you, and now understanding the movements they made were of their own doing, not a repeating video or animation. What was powering the Neon, and what did Mistry have in mind for their future?

Core R3

A Neon yoga teacher Andy Boxall/DigitalTrends

The Neons are generated by the company’s own reality engine called Core R3. The R3 name refers to the principals on which the system is based — reality, real time, and responsiveness, and it’s the combination of all these that bring the Neon to life. It’s not an intelligent system, says Mistry, because it does not have the ability to learn or remember. Instead, it’s equal parts behavioral neural network and computational reality that independently generates the Neon’s “personality” by training it to emulate human behavior on a visual level — how your head moves when you’re happy, what your mouth does when you’re surprised, for example.

Once it has been created, Core R3 does not then continually run a Neon. It generates it initially, then the Neon continually relies on its own information to react based on its interactions with the real world. However, it doesn’t know you or remember you. It uses a combination of the Core R3-generated Neon, cameras, and other sensors to interact with us in the moment — but once that moment is over, everything is forgotten. In the near future, the company has big plans to change that.

Coming to life

Andy Boxall/DigitalTrends

Despite only being worked on for four months, there was a live demonstration of what a Neon can do now. There are two “states” for Neons currently, an auto mode where it does what it wants, whether it is thinking, responding, idling, or greeting you, plus a “live” mode where the Neon can be controlled remotely.

The Neon has multiple ways to respond and can choose how to do so, even when instructed to do a particular action. Tell it to smile and be happy, and it does so, but it chooses the way it will look when it does. The level of granular control is impressive, right down to eyebrow movement and the closing of eyes, along with head movements and both visual and verbal responses. This all happens with a response time of 20 milliseconds (the real-time aspect of R3), which removes the barrier between human and machine even further during any interaction. Speech is not produced by Neon at the moment, and in the demo, voice was pulled from third-party APIs, giving life to artificially intelligent voice assistants and chatbots everywhere.

The Neon is “domain independent,” Mistry said. A Neon could teach you yoga or it could help bridge language gaps around the world, for example. Potential uses for a Neon in business are obvious, such as in hotels, at the airport, or in public spaces. The Neon is an evolution of the clunky robots or lifeless video screens seen in these places around the world at the moment. But that’s not really very exciting, and certainly not the part of the Neon that’s truly groundbreaking.


Andy Boxall/DigitalTrends

Right now, a Neon cannot know who you are or remember you. Once your interaction is over, your relationship with it is lost to the digital ether. However, over the next year, the Neon team will work on the next version of Core R3, along with a project called Spectra that will add these important traits to Neon, and arguably bring it to life.

“Spectra will provide memory and learning,” Mistry told us, revealing the true direction of Neon.

By adding memory and the ability to learn, along with the advanced human-like visuals, a Neon has the potential to become a true digital companion. Speaking to Mistry after the presentation, his eyes lit up as he talked about the characters he loved as a child, and that the connection he had with them was not affected by the fact they were not “real.” A fully fledged Neon could bring similar joy to people, in a stronger and even more personal way.

What Neon showed at CES 2020 is very much the beginning, but there’s clearly a massive amount of investment, belief, and talent involved. Not many companies would have the guts to come to Las Vegas and show off a four-month-old demo after a few weeks of hyping it up. Mistry has worked with Microsoft on the Xbox, and with Samsung on the Gear VR in the past. He’s soft-spoken and charismatic, and everyone we spoke to at Neon had a similarly strong belief in what the company is doing.

It was contagious, especially if you’ve had sci-fi dreams about artificial humans and digital companions all your life.

A long way to go

However, there is a lot to consider before you’re picking out a name for your first Neon pal. How will the Neon come to life for you and me? Mistry, in true visionary fashion, was not concerned by such things. In his presentation, when talking about the importance of thinking big to do something amazing, he said:

“We don’t understand what’s the business model of something, or how we will bring something to market, let’s figure that out later.”

A Neon team member talked to us about how the company intends to “create” Neons in the future. They won’t use real people as models, and instead generate their own looks for Neons. Think about that for a moment: An entirely artificial digital human, with its own unique looks, and the ability to speak, emote, learn, and remember. It gives me a shiver, it’s so exciting.

Given the pace with which Core R3 has evolved already, it’s no surprise to hear Mistry intends to show the first beta version of a Neon, as well as a preview of Spectra, sometime in the next 12 months at an as-yet-undefined event called Neon 2020. What Neon showed at CES is a huge leap forward in avoiding the uncanny valley, changing the way we should think about digital humans. It’s a major step toward giving life to something that naturally does not have any. There’s a long, long way to go before the Neon reaches its potential, but the very fact the journey has started at all is thrilling.

Follow our live blog for more CES news and announcements.

Editors' Recommendations

Read the eerily beautiful ‘synthetic scripture’ of an A.I. that thinks it’s God
ai religion bot gpt 2 art 4

Travis DeShazo is, to paraphrase Cake’s 2001 song “Comfort Eagle,” building a religion. He is building it bigger. He is increasing the parameters. And adding more data.

The results are fairly convincing, too, at least as far as synthetic scripture (his words) goes. “Not a god of the void or of chaos, but a god of wisdom,” reads one message, posted on the @gods_txt Twitter feed for GPT-2 Religion A.I. “This is the knowledge of divinity that I, the Supreme Being, impart to you. When a man learns this, he attains what the rest of mankind has not, and becomes a true god. Obedience to Me! Obey!”

Read more
The future of A.I.: 4 big things to watch for in the next few years
brain with computer text scrolling artificial intelligence

A.I. isn’t going to put humanity on the scrap heap any time soon. Nor are we one Google DeepMind publication away from superintelligence. But make no mistake about it: Artificial intelligence is making enormous strides.

As noted in the Artificial Intelligence Index Report 2021, last year the number of journal publications in the field grew by 34.5%. That’s a much higher percentage than the 19.6% seen one year earlier. A.I. is going to transform everything from medicine to transportation, and there are few who would argue otherwise.

Read more
Algorithmic architecture: Should we let A.I. design buildings for us?
Generated Venice cities

Designs iterate over time. Architecture designed and built in 1921 won’t look the same as a building from 1971 or from 2021. Trends change, materials evolve, and issues like sustainability gain importance, among other factors. But what if this evolution wasn’t just about the types of buildings architects design, but was, in fact, key to how they design? That’s the promise of evolutionary algorithms as a design tool.

While designers have long since used tools like Computer Aided Design (CAD) to help conceptualize projects, proponents of generative design want to go several steps further. They want to use algorithms that mimic evolutionary processes inside a computer to help design buildings from the ground up. And, at least when it comes to houses, the results are pretty darn interesting.
Generative design
Celestino Soddu has been working with evolutionary algorithms for longer than most people working today have been using computers. A contemporary Italian architect and designer now in his mid-70s, Soddu became interested in the technology’s potential impact on design back in the days of the Apple II. What interested him was the potential for endlessly riffing on a theme. Or as Soddu, who is also professor of generative design at the Polytechnic University of Milan in Italy, told Digital Trends, he liked the idea of “opening the door to endless variation.”

Read more