Skip to main content

The high-tech quest to detect COVID-19 via voice

From fever-identifying drones to 3D-printed ventilator components to the tracking of mobile location data, there’s no shortage of ways that technologists are harnessing modern tech to track (and hopefully help slow) the spread of the coronavirus pandemic.

Researchers from Carnegie Mellon University have come up with one of the most sophisticated and unlikely-sounding methods yet — but if it works it could be a game-changer for a world in which there’s a drastic shortage of proper COVID-19 testing kits. The idea? A free app that can diagnose COVID-19 simply by listening to and analyzing a user’s voiceprint.

“First and foremost, what we have here is a data-collection effort,” Bhiksha Raj, a professor at Carnegie Mellon working on the project, told Digital Trends. “After having analyzed and run preliminary tests on COVID and non-COVID voices and coughs, we believe that it may be possible to obtain a reliable flag for potential COVID by analyzing voice. We have set up a system to encourage people to contribute voice recordings. We need tens of thousands of recordings from COVID-infected subjects, and hundreds of thousands of uninfected people, both healthy and those with other problems. Our hope is that people will contribute.”

voice assistant

The app, titled the COVID Voice Detector, isn’t just collecting data for some as-yet-unspecified endpoint, however. The app is available right now for users to test out for themselves.

By visiting Carnegie Mellon’s dedicated website, users can submit to a battery of tests intended to determine their estimated “COVID-19 Score.” To gain this, they must first fill in a brief profile with information about their age, weight, sex, height, and the racial or ethnic group with which they most identify. They must also answer questions about any current cough or fever they’re currently exhibiting, along with any previous diagnosis of COVID-19. After that, it’s onto a series of audio tests such as coughing three times into your microphone, making prolonged “ooo” and “eee” sounds, and repeating the alphabet. The data is then submitted to researchers, who will use it to further hone the system. Within a minute, the user gets to find out their own results, which are presented as a sliding scale to suggest how concerned we should be.

“The COVID Score, as you currently get it, is more of a guess of how much the signatures in your voice match those of the other COVID patients we have thus far analyzed,” Raj acknowledged. “Coughs of people with infected lungs sound different. The elongated vowels like ‘aaaa’ not only sound different, but the duration to which they can be kept up is reduced. When you speak a long string of numbers, followed immediately by the alphabet, you’re being asked to speak for an extended period of time. COVID patients tend to get breathless, have runny noses, somewhat sore throats, and tire easily. All of these will affect the way they speak — particularly when they must speak for longer durations.”

But does it work?

Of course, the multi-trillion dollar question (at least, measured by the amount of money the stock market has lost since the coronavirus pandemic began) is just how well this works. As with so much about coronavirus — from questions about whether or not those who catch it develop immunity to how long our current coronavirus lockdown will last — the simple answer is that we don’t yet know.

The COVID Voice Detector team stresses that this is still an in-development tool that should not be considered as a valid alternative for an official diagnosis. “I understand that this is an experimental system which is still under development,” reads a checkbox on the website when the user creates their profile. “It is not a diagnostic system. It has not been verified by medical professionals. It is not FDA or CDC-approved, and must not be used as a substitute for a medical test or examination.”

Image used with permission by copyright holder

Nonetheless, there is some impressive talent involved. This is no fly-by-night operation. Rita Singh, a professor of computer science at Carnegie Mellon who took a lead role in developing the A.I., has long worked toward identifying micro-signatures in human voices. She believes that the voice can be used to mine a plethora of psychological, physiological, and medical data about individuals. (Other researchers have previously used the voice as a possible source of diagnostic data for diseases such as Parkinson’s.)

Meanwhile, the COVID Voice Detector’s use of a lung capacity score was created by an A.I. company called, drawing on multiple years of research, in collaboration with hospitals and doctors, to develop an accurate measure of lung capacity. (What is not clear at this stage, due to lack of data, is how this relates to COVID-19, which is known to attack the lungs.)

The availability of tools such as the COVID Voice Detector highlights a current challenge faced by the tech world. For some time, it has been evident that there is a conflict between Silicon Valley’s famous mantra of “move fast and break things” and the real world’s need for empirical proofs, proper testing, and verification. Put simply, technologists’ belief in hacking together tools and pushing them out into the world, safe in the knowledge that agile methodology will let them tweak and hone until the finished product works as well as hoped, doesn’t always mesh well with the slow, but steady, approach needed for peer review and clinical testing. This challenge is particularly pronounced in the field of medicine.

This is further tested by the coronavirus pandemic where a third factor is added in: Lack of time and resources. This scenario encourages speed-based experimentation, which must be adopted wisely to ensure that the best ideas can be rapidly deployed, without giving snake oil solutions oxygen that they don’t deserve.

Move fast and… treat things?

Which of the two solutions will the COVID Voice Detector fall into? Its own developers freely admit that there is much that still needs to be proven about efficacy. Bhiksha Raj notes that, for instance, just because someone gets a possible match score for coronavirus it does not necessarily mean that they do. There has simply not been enough analysis of COVID-19 patients to make this link clear. “I do not, by any means, want anyone to mistake it for professional or semi-professional opinion and take healthcare decisions based on [this],” Raj said. “If they did, they could endanger themselves and others.”

This is, at best, currently a triage tool that could prompt people to seek out further medical opinions. But if it functions as hoped, the potential upside could be huge.

“If this works, we will have a very simple and easy way of monitoring millions of people,” Raj said. “Not only can we get instantaneous evaluations, but also look at longitudinal trends among subjects who use it repeatedly. This could provide a way tracking health outbreaks in general in future — particularly ones that affect voice.”

For this reason, Carnegie Mellon plans to share the data it gathers with other researchers around the world, encouraging them to work on it either collaboratively or independently. The one catch? “We will need the assurance from anyone who uses our data that they will not be protecting or commercializing any IP that comes out of it either.”

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Worried about bills during the coronavirus pandemic? This A.I. lawyer can help
Paying bills 1

Anyone who says they’re not worried about the future right now isn’t telling the truth. But while we’re all worried about the health implications of the COVID-19 coronavirus, for many the economic ramifications run an extremely close second. Especially for those without savings, or those in low-paid gig economy jobs, anxieties about how to pay the next bill are pretty darn scary. Even if it’s a type of scary that no Hollywood pandemic movie is ever going to touch.

Give a round of applause, then, to Joshua Browder: The 24-year-old legal-tech whiz kid behind DoNotPay, a growing arsenal of free automated A.I. tools that everyday consumers can use to do everything from disputing parking tickets to suing robocall scammers. His latest creation is tailor-made for those fretting about the impact that the coronavirus crisis is going to have on their ability to pay the next time they face demands.

Read more
Thermal AR glasses give screening for possible coronavirus a high-tech twist
M400 on Helmet

Introducing Vuzix M4000 Smart Glasses at CES2020

Smartglasses can have all sorts of handy use-cases, from augmented reality gaming to bringing up contextual information about objects or surroundings to help the wearer. In the age of the COVID-19 novel coronavirus, they might have another useful application, however; assisting people in helping spot possible signs of the virus from a distance. This could be used to help border control officials and health workers as they come to grips with the most serious public health crisis in recent years.

Read more
Mayflower Autonomous Ship is headed to sea to test its self-driving boat tech
mayflower ship seat tests ibm

Entering the Mind of the Mayflower

Last year, Digital Trends wrote about an ambitious project that seeks to dispatch an unmanned autonomous ship across the Atlantic Ocean on the 400th anniversary of the voyage of the Mayflower. While this first-of-its-kind journey won’t take place until the this fall, IBM and marine research organization Promare announced Thursday that the “A.I. Captain” that will power the self-steering vessel is ready to go to sea for a month of testing. This trial, which will take place on a manned research vessel off the coast of Plymouth in the U.K., will test out the onboard A.I. and edge computing system to see how well it navigates.

Read more