Skip to main content

AI assistants will soon recognize and respond to the emotion in your voice

You know when people say that it’s not what you say, but how you say it that matters? Well, very soon that could become a part of smart assistants such as Amazon’s Alexa or Apple’s Siri. At least, it could if these companies decide to use new technology developed by emotion tracking artificial intelligence company Affectiva.

Affectiva’s work has previously focused on identifying emotion in images by observing the way that a person’s face changes when they express particular sentiments. Affectiva’s latest technology builds on that premise through the creation of a cloud-based application program interface (API) that is able to detect emotion in speech. Developed using the power of deep learning technology, the smart tech is capable of observing changes in tone, volume, speed, and voice quality and using this to recognize emotions like anger, laughter, and arousal in recorded speech.

Recommended Videos

“The addition of Emotion AI for speech builds on Affectiva’s existing emotion recognition technology for facial expressions, making us the first AI company to allow for a person’s emotions to be measured across face and speech,” Rana el Kaliouby, co-founder and CEO of Affectiva, told Digital Trends. “This is all part of a larger vision that we have. People sense and express emotion in many different ways: Through facial expressions, voice, and gestures. We’ve set out to develop multi-modal Emotion AI that can detect emotion the way humans do from multiple communication channels. The launch of Emotion AI for speech takes us one step closer.”

Affectiva Overview

Affectiva developed its voice recognition system by collecting naturalistic speech data from a variety of sources, including commercially available databases. This data was then labeled by human experts for the occurrence of what the company calls “emotion events.” These human generated labels were used to train and validate the team’s deep learning models, so that over time it grew to understand how certain shifts in a person’s voice might indicate a particular emotion.

It’s smart stuff from a technology perspective but, like the best technology, it also has the possibility of helping users on a practical basis. One specific application could include car navigation systems that are able to hear a driver start to experience road rage, and react to prevent them from making a rash driving decision. It could similarly be used to allow automated assistants to change their approach when they hear anger or frustration from a user — or to learn what kind of responses elicit the best reactions and repeat these strategies.

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Can AI really replace your keyboard and mouse?
Gemini Live on a phone atop a keyboard.

“Hey ChatGPT, left-click on the enter password field in the pop-up window appearing in the lower left quadrant of the screen and fill XUS&(#($J, and press Enter.” 

Fun, eh? No, thanks. I'll just move my cheap mouse and type the 12 characters on my needlessly clicky keyboard, instead of speaking the password out loud in my co-working space.

Read more
AI could soon speak dog and cat
A cute dog.

Imagine what it would be like to know exactly what your dog was saying when it barked, or your cat when it miaowed, or your iguana when it ... made whatever sound an iguana makes (do they even make one?).

There have been various attempts to develop systems that enable pets to communicate -- a notable example is where dogs press programmable sound buttons, with each one representing a word or phrase, to "talk" and express needs, emotions, or observations -- but none are quite so exciting as the one proposed by Chinese tech giant Baidu in a patent filed with the China National Intellectual Property Administration.

Read more
AI enables slain man to address courtroom at killer’s sentencing
An abstract image of a person and a blue background.

In what’s believed to be a world first, artificial intelligence (AI) has allowed a slain man to address his killer at the sentencing hearing.

Christopher Pelkey was shot dead in a road rage incident in Chandler, Arizona, four years ago, but just recently, AI was used to recreate a digital version of the victim that was allowed to make a statement during the trial, a local news site reported.

Read more