Skip to main content

Software upgrade could let smartwatches know exactly what our hands are doing

Sensing Fine-Grained Hand Activity with Smartwatches

Smartwatches are increasingly becoming their own thing, rather than simply accessories which rely on and mirror the functionality of other devices. However, there are some scenarios in which it would be useful for our smartwatches to work more closely with the objects around us — whether that’s knowing what we’re doing on our smartphones, working out when we’re typing on a keyboard, or even something as mundane as understanding when we’re using a cheese grater or washing our hands under a tap.

To solve this problem, researchers from Carnegie Mellon University figured out a way to capture fine-grained hand activity in smartwatch wearers. The results could make the experience of using a smartwatch more powerful and contextually aware.

“For so long, smartwatches have only been able to track whole-body activities, like walking, running, cycling and sleeping,” Chris Harrison, head of Carnegie Mellon’s Future Interfaces Group (FIG), told Digital Trends. “But with smartwatches right there on the wrist, we’ve always suspected it would be possible for them to also track what the hands are doing too, like typing, writing, eating, drinking, scratching, [and more]. This opens up a whole new world of high fidelity activity recognition that wasn’t previously possible.”

In a paper being presented this week at the ACM CHI 2019 conference in Glasgow, Scotland, the researchers describe how they were able to achieve this fine-grained hand activity data by overclocking the smartwatch’s in-built accelerometer to 4kHz. Combining data gathered from volunteers with the right machine learning algorithms, the system is able to determine which one of 25 hand activities is being carried out at any given time — with 95.2% accuracy.

“That’s the best part: [it requires] no extra hardware,” Harrison continues. “Your smartwatch already has everything it needs to unlock this type of tracking. It could be enabled on devices with little more than a software update.”

Harrison and co-author Gierad Laput describe a number of different potential applications. For example, by tracking when you are typing, your smartwatch could recommend regular breaks. It could also track often you drink, thereby reminding you to stay hydrated. Similarly, it could be used to track how often and long a person eats for, something which could be utilized for rudimentary calorie counting. In addition, it could detect when a person fidgets with their hands, and use this as a proxy for anxiousness to build systems that are more responsive to your mood.

“If we can sense what your hands are doing, it could potentially unlock a set of compelling applications that are more assistive, more accommodating, and can lead people to live healthier lives,” Harrison said.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
This AI cloned my voice using just three minutes of audio
acapela group voice cloning ad

There's a scene in Mission Impossible 3 that you might recall. In it, our hero Ethan Hunt (Tom Cruise) tackles the movie's villain, holds him at gunpoint, and forces him to read a bizarre series of sentences aloud.

"The pleasure of Busby's company is what I most enjoy," he reluctantly reads. "He put a tack on Miss Yancy's chair, and she called him a horrible boy. At the end of the month, he was flinging two kittens across the width of the room ..."

Read more
Digital Trends’ Top Tech of CES 2023 Awards
Best of CES 2023 Awards Our Top Tech from the Show Feature

Let there be no doubt: CES isn’t just alive in 2023; it’s thriving. Take one glance at the taxi gridlock outside the Las Vegas Convention Center and it’s evident that two quiet COVID years didn’t kill the world’s desire for an overcrowded in-person tech extravaganza -- they just built up a ravenous demand.

From VR to AI, eVTOLs and QD-OLED, the acronyms were flying and fresh technologies populated every corner of the show floor, and even the parking lot. So naturally, we poked, prodded, and tried on everything we could. They weren’t all revolutionary. But they didn’t have to be. We’ve watched enough waves of “game-changing” technologies that never quite arrive to know that sometimes it’s the little tweaks that really count.

Read more
Digital Trends’ Tech For Change CES 2023 Awards
Digital Trends CES 2023 Tech For Change Award Winners Feature

CES is more than just a neon-drenched show-and-tell session for the world’s biggest tech manufacturers. More and more, it’s also a place where companies showcase innovations that could truly make the world a better place — and at CES 2023, this type of tech was on full display. We saw everything from accessibility-minded PS5 controllers to pedal-powered smart desks. But of all the amazing innovations on display this year, these three impressed us the most:

Samsung's Relumino Mode
Across the globe, roughly 300 million people suffer from moderate to severe vision loss, and generally speaking, most TVs don’t take that into account. So in an effort to make television more accessible and enjoyable for those millions of people suffering from impaired vision, Samsung is adding a new picture mode to many of its new TVs.
[CES 2023] Relumino Mode: Innovation for every need | Samsung
Relumino Mode, as it’s called, works by adding a bunch of different visual filters to the picture simultaneously. Outlines of people and objects on screen are highlighted, the contrast and brightness of the overall picture are cranked up, and extra sharpness is applied to everything. The resulting video would likely look strange to people with normal vision, but for folks with low vision, it should look clearer and closer to "normal" than it otherwise would.
Excitingly, since Relumino Mode is ultimately just a clever software trick, this technology could theoretically be pushed out via a software update and installed on millions of existing Samsung TVs -- not just new and recently purchased ones.

Read more