Smartwatches are increasingly becoming their own thing, rather than simply accessories which rely on and mirror the functionality of other devices. However, there are some scenarios in which it would be useful for our smartwatches to work more closely with the objects around us — whether that’s knowing what we’re doing on our smartphones, working out when we’re typing on a keyboard, or even something as mundane as understanding when we’re using a cheese grater or washing our hands under a tap.
To solve this problem, researchers from Carnegie Mellon University figured out a way to capture fine-grained hand activity in smartwatch wearers. The results could make the experience of using a smartwatch more powerful and contextually aware.
“For so long, smartwatches have only been able to track whole-body activities, like walking, running, cycling and sleeping,” Chris Harrison, head of Carnegie Mellon’s Future Interfaces Group (FIG), told Digital Trends. “But with smartwatches right there on the wrist, we’ve always suspected it would be possible for them to also track what the hands are doing too, like typing, writing, eating, drinking, scratching, [and more]. This opens up a whole new world of high fidelity activity recognition that wasn’t previously possible.”
In a paper being presented this week at the ACM CHI 2019 conference in Glasgow, Scotland, the researchers describe how they were able to achieve this fine-grained hand activity data by overclocking the smartwatch’s in-built accelerometer to 4kHz. Combining data gathered from volunteers with the right machine learning algorithms, the system is able to determine which one of 25 hand activities is being carried out at any given time — with 95.2% accuracy.
“That’s the best part: [it requires] no extra hardware,” Harrison continues. “Your smartwatch already has everything it needs to unlock this type of tracking. It could be enabled on devices with little more than a software update.”
Harrison and co-author Gierad Laput describe a number of different potential applications. For example, by tracking when you are typing, your smartwatch could recommend regular breaks. It could also track often you drink, thereby reminding you to stay hydrated. Similarly, it could be used to track how often and long a person eats for, something which could be utilized for rudimentary calorie counting. In addition, it could detect when a person fidgets with their hands, and use this as a proxy for anxiousness to build systems that are more responsive to your mood.
“If we can sense what your hands are doing, it could potentially unlock a set of compelling applications that are more assistive, more accommodating, and can lead people to live healthier lives,” Harrison said.
