Skip to main content

Samsung tapped Hollywood talent to bring AR Emoji to life

 

Embedded into the Samsung Galaxy S9’s camera app, AR Emoji allows you to use the front-facing camera to create and send digital cartoon avatars. Similar to Apple’s Animojis in concept, AR Emoji captures your facial movements and mirrors them in the form of a character or animal. But with its “Create My Emoji” feature, you can also create one of yourself by simply taking a selfie — allowing you to choose a skin tone, outfit, hairstyle, hair color, and glasses.

Recommended Videos

With AR Emoji, Samsung set out to solve what it sees as a problem — finding a way to replace existing emojis with ones that are interactive. The company wanted to do this in a way that was simple, by providing users with the right tools to capture emotion but still allowed them to easily share the result on a variety of messaging platforms.

So, how did Samsung create its AR Emojis? For starters, it went straight to the 3D animation experts who brought to life some of Hollywood’s most popular characters.

From big screen to smartphone camera

To power the new feature, Samsung collaborated with Loom.ai —  a San Francisco-based startup specializing in 3D animation. The company’s software allows users to create fully personalized and customizable 3D avatars from a single photograph, without custom hardware or depth sensors.Through a partnership and a software development kit (SDK), Samsung combined Loom.ai’s technology with the S9’s camera tech (augmented reality and facial recognition) to make custom AR Emoji possible.

Samsung wants to find a way to replace existing emojis with ones that are interactive.

The vision behind Loom.ai all started with movies — cartoons and superheros to be exact. Having previously worked at popular visual effects companies like DreamWorks and Lucasfilm, Loom.ai’s founders –Mahesh Ramasubramanian and Kiran Bhat — have helped bring well-known characters, such as Shrek and the Hulk, to the big screen.

To inject personality into those characters, they used performance-capture technology to incorporate the actors who played them — recording the movements of the actor, not their visual appearance. But after seeing how movie audiences reacted to their 3D creations, Ramasubramanian and Bhat decided to take that technology further — beyond Hollywood — by starting Loom.ai, which focuses on smartphones.

“People instantly react to having these characters come to life when it’s driven by human actors,” Ramasubramanian, who serves as CEO and co-founder, told Digital Trends. “We started Loom.ai with the goal that we need to be able to find an easy way for every individual to be able to see themselves in 3D, and for that we had to build a new technology.”

AR Emoji
Brenda Stolyar/Digital Trends

With feature films, creating a 3D avatar requires tons of equipment — expensive hardware and software, makeup, and a studio setting. But Loom.ai found a way to simplify the process of capturing a person’s features without the usual tools; instead, its patented technology uses a single photograph and deep learning to analyze an image. If a photograph is free of any facial expressions, the deep-learning algorithm looks at the pixels and automatically produces a face that encapsulates the correct muscle movement.

“When Samsung’s partnership rolled around we had a chance to implement that sophisticated technology on a device, which we feel is already a pretty compelling engineering task by itself,” Bhat, Loom.ai’s CTO and co-founder, told Digital Trends. “It’s a big technology jump in being able to take some of those ideas and aspirations from the big screen and bring it to mobile.”

To power AR Emoji, Samsung enlisted the help of Loom.ai —  a San Francisco-based startup specializing in 3D animation.

Incorporating this technology into the Galaxy S9 hardware required Loom.ai to restructure its algorithms, in order to run on the device. But Loom.ai did not create AR Emoji alone: Both Ramasubramanian and Bhat credit Samsung in dedicating tremendous effort into optimizing the software. By providing the SDK to integrate into smartphones, the Loom.ai founders think Samsung can make the framework much richer by adding more content and customizability options in the future.

How the Galaxy S9 creates a digital avatar

So, how exactly does the S9 (and S9 Plus) create AR Emojis? While Samsung didn’t reveal its secret sauce, it gave Digital Trends a look under the hood.

Working off Loom.ai’s technology, the phone’s camera captures and processes a 2D image into 3D. To enhance this further, the S9 uses additional software — facial tracking and Samsung’s Blend Shape technology — to provide facial expressions for the AR Emoji.

You’re then left with a cartoon-like version of yourself that can emote 18 different expressions — which Samsung plans on slowly expanding to 54 — ranging from joy and pleasure, to sorrow and anger. All data is also local to the device, and doesn’t get stored anywhere else.

Both working together, facial tracking technology tracks over 100 facial features in real time (such as winking or smiling) to capture individual emotions, while Blend Shape helps to deliver a greater range of them — which blends expressions together based on 75 basic human facial features. Samsung says, for Blend Shape, it conducted at least 100 tests on a weekly basis within its labs, in order to make sure there is enough data that would entice its users. (Blend Shape is also used to create its Disney AR Emojis, like Mickey and Minnie Mouse.)

In our own AR Emoji test, a majority of us felt our characters looked extremely inaccurate. One writer felt there was a lack of skin-tone options available, with no dark-skinned tones to choose from. Others felt their AR Emoji didn’t look anything like them, which made editing the avatar seem like more of a chore than an enjoyable task.

According to Loom.ai, creating an accurate AR Emoji relies heavily on the way the selfie is shot. Users should go about taking the photo the same way they’d take a passport photo. “You want even lighting, nothing covering your face, and a neutral expression so that we can add the expressions later,” Ramasubramanian explained. “It’s a matter of people getting a little accustomed to it and then seeing the possibilities of animating it and using it.”

Ramasubramanian also said that users should treat it the way they would while taking any selfie — meaning, you can take tons of photos until you’re satisfied with the photo. Since AR Emoji allows you to create multiple avatars, you can compare them to find the one you like best. Each one you create also saves to your phone, so you can switch up the version you use depending on your mood.

The latest update to AR Emoji includes additional hairstyles, headwear such as a hijab, and other options. The update is currently rolling out to unlocked versions of the Samsung Galaxy S9 and S9 Plus in Korea, China, and Europe. European carrier devices and U.S. unlocked devices should have the update available for download this week. The update for U.S. carrier devices will be rolling out over the coming weeks. Samsung says AR Emoji will continue to evolve in the future, but for now, is staying tight lipped on exact details.

Brenda Stolyar
Former Staff Writer, Mobile
Brenda became obsessed with technology after receiving her first Dell computer from her grandpa in the second grade. While…
Samsung opens One UI 7 beta for foldables and confirms stable release window
Samsung Galaxy AI on the Galaxy Z Fold 6 and Galaxy Z Flip 6.

Samsung kicked off the beta test program for Android 15-based One UI 7 software in December. But so far, it has remained exclusive to a handful of devices and with no final release date in sight. That status-quo has finally changed today.

The company has confirmed that the stable build of One UI 7 will be released in April. In addition to it, more devices will be added to the beta testing program starting tomorrow for users based in India, South Korea, the UK, and the US.

Read more
I love the Clicks iPhone keyboard, and it’s now coming to Android
Clicks for Moto Razr+ 2024 in hand

Do you remember the world of BlackBerry? The precision with which we typed, the endless keyboard shortcuts, and the satisfying haptic feedback from pressing a button and seeing something dance across the page?

Even if you don’t, you may have found that you type much better at a computer than you do on your phone, especially if you use one of the best phones with a small screen. Last January, a new startup from London — featuring some very familiar faces — launched a new iPhone accessory that brought the buttons back.

Read more
How good is the Galaxy S25 Ultra’s new ultra-wide camera? I tested it to find out
The Samsung Galaxy S25 Ultra's camera.

Samsung’s approach to the camera in its Ultra phones has focused on evolution and introducing incremental improvements with each subsequent model.

Last year’s Galaxy S24 Ultra saw Samsung switch the secondary telephoto lens to a 50MP sensor with 5x optical zoom. Higher resolution and a higher pixel count thanks to pixel binning meant the zoom quality was improved over the Galaxy S23 Ultra’s 10x zoom, despite a lower optical zoom length. The Galaxy S25 Ultra also features the 200MP main camera that made its debut in the Galaxy S23 Ultra.

Read more