“This is a matter of building the next touchscreen, “said Ohto Pentikäinen, CEO of a company called Doublepoint, as he introduced Digital Trends to its fascinating, next-generation gesture control system. The best way you could think of it is as Apple’s Double Tap feature on steroids.
I chatted with Pentikäinen over a video call during CES 2024 to learn more.
More than an app
You may have seen the app Doublepoint launched for Wear OS, called Wow Mouse, at CES 2024 already, but it’s only the beginning of what the company is working on. Wow Mouse adds gesture control to your Wear OS smartwatch, effectively turning it into a mouse on your wrist that uses finger and wrist movements to operate. But Wow Mouse is a relatively basic demonstration of what Doublepoint is working on.
“We provide something a lot more comprehensive,” Pentikäinen said when asked about the Wow Mouse app, “and there’s a lot more we can do than just a single tap. The biggest thing that our current hardware is capable of doing is pinch and hold, which means you can actually scroll, as well as manipulate objects.”
Doublepoint calls these ‘micro gestures,’ and they include everything from tap to select, tap to scroll, and double taps. Having additional gestures that go beyond Double Tap’s double-pinch gesture is very appealing and gets to the heart of why Pentikäinen compared the technology to a touchscreen, as it replicates many of the same gestures we use on them already.
So, how does Doublepoint’s system work?
How these gestures work
“How micro gesture detection works is unlike computer vision-based hand tracking,” Pentikäinen explained, using a familiar comparison. “That uses the camera to calculate the pose of your hand, and it does this every millisecond or whatever the frame rate is when your fingers touch each other. The way we detect touch uses the Inertial Measurement Unit.”
The Inertial Measurement Unit (IMU) sounds like a special piece of hardware, but in fact, it simply includes an accelerometer, gyroscope, and magnetometer. In other words, core sensors that are found in almost all smartwatches and other wearables.
“It looks for a vibration caused when your fingers touch each other,” Pentikäinen said, at which point Doublepoint’s algorithms then differentiate that vibration from every other vibration that occurs when you move your fingers. However, the innovative pinch-and-hold gesture, which is very unusual and critical to the algorithm’s appeal, is a little different and far more complicated.
“For pinch and hold, we also need to know when your fingers are kept together, and we actually use tendon monitoring for that. We use the optical heart rate sensor not to look at muscles, but the tendons that connect your finger to the muscle, and it’s the movement that tells us when you stop touching,” he said.
Configuration and accuracy
Part of the success of Apple’s Double Tap is that it works without any setup or configuration, and is also highly accurate. What about Doublepoint’s system?
“We wanted to make it one of our selling points from the very beginning that we’re not going to need any configuration, and when you put it on, it works instantly. What that means is we need to have a process in place to collect data from a wide variety of people and take into account relevant factors like Body Mass Index (BMI), age, and melatonin levels. We’ve been able to get to very, very good accuracy levels. Almost in the 97% to 98% range. Of course, there’s still a lot of work to do,” he said.
What’s also interesting is a smartwatch with Doublepoint’s gesture recognition can be paired with other Bluetooth devices, such as those with eye tracking, so they work together. It makes controlling systems like this more natural and more private, too, as there is no need for voice recognition as a backup or alternative.
Qualcomm is interested
Gesture controls are nothing new, but there’s no doubt Apple’s Double Tap system has reignited interest in them recently. And it’s likely to only get more attention once the Vision Pro headset is released, as other competing systems may try new and different things to make controlling headsets easier and more intuitive. It’s, therefore, probably no surprise to hear Qualcomm is interested in Doublepoint’s technology.
“We do have a vendor agreement with Qualcomm,” Pentikäinen revealed. “It means the idea is to embed our algorithms into Qualcomm chips, and they, of course, provide these chips to a lot of different watch and fitness tracker providers out there today.”
In addition to Qualcomm, it’s also working with sensor companies to embed its algorithm in the sensor itself. Then there is the company’s own reference smartwatch platform, which Pentikäinen said “is designed to give people an idea of what we can do with [the algorithms], and show how we would build it into a product.”
Coming soon?
Does this mean a new smartwatch with gesture and pinch-and-hold technology from Doublepoint that allows you to scroll, swipe, alter values like the volume or brightness is coming soon?
“We don’t have confirmed schedules on when our partners are able to take their products to market, but I would say these things are currently evolving very quickly,” Pentikäinen said.
Several times during our brief conversation, Pentikäinen made comparisons between Doublepoint’s gesture control system and the touchscreen, and we finished with a better explanation of why he made them.
“The touchscreen started out from the phone, basically, and now we see it everywhere,” he said. “It enabled all sorts of new devices. Apple doubled down on gesture recognition last year, and they basically showed that gestures are going to be a vital way of improving. So for us, we just want to enable this for as many companies out there as possible, using what we think is the next touchscreen.”
While we will have to wait for a smartwatch with Doublepoint’s technology to arrive, you can try out a basic version through the Wow Mouse app, which is ready to download through Google Play for selected Wear OS smartwatches right now.
