Skip to main content

Samsung tapped Hollywood talent to bring AR Emoji to life

Embedded into the Samsung Galaxy S9’s camera app, AR Emoji allows you to use the front-facing camera to create and send digital cartoon avatars. Similar to Apple’s Animojis in concept, AR Emoji captures your facial movements and mirrors them in the form of a character or animal. But with its “Create My Emoji” feature, you can also create one of yourself by simply taking a selfie — allowing you to choose a skin tone, outfit, hairstyle, hair color, and glasses.

With AR Emoji, Samsung set out to solve what it sees as a problem — finding a way to replace existing emojis with ones that are interactive. The company wanted to do this in a way that was simple, by providing users with the right tools to capture emotion but still allowed them to easily share the result on a variety of messaging platforms.

So, how did Samsung create its AR Emojis? For starters, it went straight to the 3D animation experts who brought to life some of Hollywood’s most popular characters.

From big screen to smartphone camera

To power the new feature, Samsung collaborated with —  a San Francisco-based startup specializing in 3D animation. The company’s software allows users to create fully personalized and customizable 3D avatars from a single photograph, without custom hardware or depth sensors.Through a partnership and a software development kit (SDK), Samsung combined’s technology with the S9’s camera tech (augmented reality and facial recognition) to make custom AR Emoji possible.

Samsung wants to find a way to replace existing emojis with ones that are interactive.

The vision behind all started with movies — cartoons and superheros to be exact. Having previously worked at popular visual effects companies like DreamWorks and Lucasfilm,’s founders –Mahesh Ramasubramanian and Kiran Bhat — have helped bring well-known characters, such as Shrek and the Hulk, to the big screen.

To inject personality into those characters, they used performance-capture technology to incorporate the actors who played them — recording the movements of the actor, not their visual appearance. But after seeing how movie audiences reacted to their 3D creations, Ramasubramanian and Bhat decided to take that technology further — beyond Hollywood — by starting, which focuses on smartphones.

“People instantly react to having these characters come to life when it’s driven by human actors,” Ramasubramanian, who serves as CEO and co-founder, told Digital Trends. “We started with the goal that we need to be able to find an easy way for every individual to be able to see themselves in 3D, and for that we had to build a new technology.”

AR Emoji
Brenda Stolyar/Digital Trends

With feature films, creating a 3D avatar requires tons of equipment — expensive hardware and software, makeup, and a studio setting. But found a way to simplify the process of capturing a person’s features without the usual tools; instead, its patented technology uses a single photograph and deep learning to analyze an image. If a photograph is free of any facial expressions, the deep-learning algorithm looks at the pixels and automatically produces a face that encapsulates the correct muscle movement.

“When Samsung’s partnership rolled around we had a chance to implement that sophisticated technology on a device, which we feel is already a pretty compelling engineering task by itself,” Bhat,’s CTO and co-founder, told Digital Trends. “It’s a big technology jump in being able to take some of those ideas and aspirations from the big screen and bring it to mobile.”

To power AR Emoji, Samsung enlisted the help of —  a San Francisco-based startup specializing in 3D animation.

Incorporating this technology into the Galaxy S9 hardware required to restructure its algorithms, in order to run on the device. But did not create AR Emoji alone: Both Ramasubramanian and Bhat credit Samsung in dedicating tremendous effort into optimizing the software. By providing the SDK to integrate into smartphones, the founders think Samsung can make the framework much richer by adding more content and customizability options in the future.

How the Galaxy S9 creates a digital avatar

So, how exactly does the S9 (and S9 Plus) create AR Emojis? While Samsung didn’t reveal its secret sauce, it gave Digital Trends a look under the hood.

Working off’s technology, the phone’s camera captures and processes a 2D image into 3D. To enhance this further, the S9 uses additional software — facial tracking and Samsung’s Blend Shape technology — to provide facial expressions for the AR Emoji.

You’re then left with a cartoon-like version of yourself that can emote 18 different expressions — which Samsung plans on slowly expanding to 54 — ranging from joy and pleasure, to sorrow and anger. All data is also local to the device, and doesn’t get stored anywhere else.

Both working together, facial tracking technology tracks over 100 facial features in real time (such as winking or smiling) to capture individual emotions, while Blend Shape helps to deliver a greater range of them — which blends expressions together based on 75 basic human facial features. Samsung says, for Blend Shape, it conducted at least 100 tests on a weekly basis within its labs, in order to make sure there is enough data that would entice its users. (Blend Shape is also used to create its Disney AR Emojis, like Mickey and Minnie Mouse.)

In our own AR Emoji test, a majority of us felt our characters looked extremely inaccurate. One writer felt there was a lack of skin-tone options available, with no dark-skinned tones to choose from. Others felt their AR Emoji didn’t look anything like them, which made editing the avatar seem like more of a chore than an enjoyable task.

According to, creating an accurate AR Emoji relies heavily on the way the selfie is shot. Users should go about taking the photo the same way they’d take a passport photo. “You want even lighting, nothing covering your face, and a neutral expression so that we can add the expressions later,” Ramasubramanian explained. “It’s a matter of people getting a little accustomed to it and then seeing the possibilities of animating it and using it.”

Ramasubramanian also said that users should treat it the way they would while taking any selfie — meaning, you can take tons of photos until you’re satisfied with the photo. Since AR Emoji allows you to create multiple avatars, you can compare them to find the one you like best. Each one you create also saves to your phone, so you can switch up the version you use depending on your mood.

The latest update to AR Emoji includes additional hairstyles, headwear such as a hijab, and other options. The update is currently rolling out to unlocked versions of the Samsung Galaxy S9 and S9 Plus in Korea, China, and Europe. European carrier devices and U.S. unlocked devices should have the update available for download this week. The update for U.S. carrier devices will be rolling out over the coming weeks. Samsung says AR Emoji will continue to evolve in the future, but for now, is staying tight lipped on exact details.

Editors' Recommendations

Brenda Stolyar
Former Digital Trends Contributor
Brenda became obsessed with technology after receiving her first Dell computer from her grandpa in the second grade. While…
Samsung just announced 3 new Android tablets. Here’s a first look
Someone holding the Samsung Galaxy Tab S9 Plus.

Samsung has a fresh portfolio of tablets to take on the iPad lineup. These flagship Android slates span three size formats and price points. The new Galaxy Tab S9 trio is nearly identical in heft and footprint to the Galaxy Tab S8 series, save for a minor design tweak, a new chip, and fresh color options.

The pill-shaped camera island is gone, and we now have a floating lens look that's similar to that of the Galaxy S23 series phones. Plus, there’s an eye-pleasing beige color option to pick from this year. The rest of the package is a modest iterative upgrade at best.
Galaxy Tab S9 series: specs and features

Read more
Samsung just gave us 3 big Galaxy Unpacked teasers
Samsung Galaxy Z Fold 4 and Galaxy Z Flip 4.

Samsung's next Galaxy Unpacked is almost here, and in typical Samsung fashion, the company is offering a small glimpse of what we can expect from the festivities next week.

TM Roh, the current head of Samsung's Mobile division, just shared his pre-Unpacked letter. While it's mostly marketing hype to get Samsung fans excited about next week's Unpacked event, there are a few teases that give us a small glimpse of what to expect.
Samsung's foldables are getting lighter

Read more
Here’s everything we expect from Samsung’s next Unpacked event
Official invite for Samsung's July 2023 Unpacked event.

It’s official — Samsung’s next Galaxy Unpacked event will be on July 26, 2023, in Seoul, Korea. And boy, this will be a jam-packed event. We’re going to see the next generation of Samsung's foldables, smartwatches, and tablets.  So, let’s break it all down.

Here’s everything we’re expecting to see at the Samsung Galaxy Unpacked July 2023 event — from the Galaxy Z Fold 5, new Galaxy Watch 6, and everything in between.
Samsung Galaxy Z Fold 5

Read more