Need a ruler for your tablet? Qeexo conjures virtual tools with natural gestures

“Everybody uses pinch and zoom, because it’s easy to remember,” Sang Won Lee, CEO of a gesture control company called Qeexo told Digital Trends. “The reason we don’t use three- or four-finger gestures is because we can’t remember what they do — but you can probably hold a computer mouse with your eyes closed, right?”

Absolutely. Try making that shape with your hand right now, and we’re pretty sure you’ll succeed. Now imagine using that same gesture to summon up a virtual mouse on the screen of your tablet; or in the future, an interactive tabletop, complete with left and right buttons, and that always-helpful scroll wheel. Such natural movements could transform the way we interact with touchscreens, especially large ones.

Qeexo can make those kinds of gestures possible right now, and we got a chance to see its impressive technology in action.

Machine learning

Anyone who has used a modern Huawei phone may be familiar with Qeexo’s first product. It’s called Finger Sense, but Huawei changed the name to Knuckle Sense for use in its Mate S, Mate 8, the P9, and other phones. The concept is simple. You use your knuckle instead of a fingertip to activate various features, such as taking a screenshot or opening apps. Underneath Finger Sense is a versatile machine learning engine, around which Qeexo has built Touch Tools and Impact Sense: two tools that go way beyond using just our knuckle.

“We don’t want to make people learn a whole lot of new gestures, we want to make gestures that people are already familiar with.”

The mouse gesture is only the start for Touch Tools. It can also generate a pen, pencil, or paint brush, then move on to erasers of all sizes, a magnifying glass, a tape measure, a ruler, and even a camera for taking quick screenshots. The ability to conjure helpful tools on demand with simple gestures would certainly make our lives easier and limit the number of accessories and third-party apps we buy to meet our needs. Qeexo puts all the functionality of old-school real-world tools quite literally at the tips of our fingers.

For the last couple of years, Qeexo has refined its machine learning code to teach our devices how we, as humans, actually wield and use tools. The company has demonstrated the tech in other products such as FingerAngle along the way, and it’s the real star of the show. You simply teach the machines the way you personally hold a mouse or a pen, so that it always interprets the gesture correctly. It doesn’t just guess what you’re doing based on what hundreds of other people do, plus it will adapt to changes in the way you “make” the mouse, camera, or any other gesture over time. Adding this kind of intelligence increases the likelihood that the machine will correctly guess which type of gesture you made to provide you with the tool you want.

Beyond mobile

We watched Qeexo’s Touch Tools work on an iPad, and it looked great. It was a super slick demo that made me grin, just like all the best tech innovations should. However, it gets really exciting when you start to consider its use outside of a mobile device. The gestures seen here can be adapted to any shape and size, so could be made into a knob or dial, ready for use on a touchscreen inside a car.

For instance, a large touchscreen dominates the Tesla Model S’s dashboard, and although it looks amazing, it’s a problem to use on the move because you have to look at the display to find settings and make adjustments. That’s not a good idea when you’re moving along. Now imagine reaching out and grabbing a virtual dial to adjust the volume of the stereo or turn down the heat. For Touch Tools, this is simple; and for the driver, it’s a potential life saver.

“We don’t want to make people learn a whole lot of new gestures, we want to make gestures that people are already familiar with,” Sang continued. But how would we differentiate between dials, and understand we’re grabbing the right one without looking? “The button for the volume in my car is smaller than the one for the AC, so you’d make a different gesture, but what we can also do is link the gesture to haptic feedback, and as soon as you turn the dial you’ll feel that the engine has activated the right tool.”

Because conflicts could arise between similarities in gestures, the company will work with app developers to integrate only the most relevant tools for the situation. We don’t think anyone will need a magnifying glass in the car, for example. Qeexo’s keen to avoid gesture overload, which is one of the problems it wants to solve.

Impact Force

Impact Force (which sounds like the title of another Dirty Harry movie, but isn’t) is a little different to Touch Tools, but uses the same learning engine. It’s designed as an alternative to pressure-sensitive gesture systems, like 3D Touch on the iPhone 6S and the Apple Watch.

“People are struggling to trigger force sense functionality. They can’t tell the difference between a long press and a hard press,” according to Sang, “because we don’t have a pressure sensor on the end of our finger.” Qeexo sees value in the concept of a third selection process, but also wants to solve it on a software level, making it more accessible to manufacturers that don’t only produce high-end, expensive smartphones.

Impact Force reacts with a sharp impact on the display, and the machine learning engine will be able to understand the difference between a tap, a long press, and a hard/sharp press. It’s an alternative to changing out hardware to measure pressure, but there is a difference between the two approaches. “We can do most of the things a pressure sensitive screen does, but not continuous pressure sensing, such as that used for Huawei’s weight measurement trick.”

Coming soon

Why haven’t we seen this approach to gesture controls before now?

“It took us two and a half years to perfect,” said Sang. “We had to do a lot of optimization work based on feedback from customers, such as bringing down the power consumption, and get the CPU usage under a certain limit.”

Geexo Gesture
Andy Boxall/Digital Trends
Andy Boxall/Digital Trends

Now it’s ready for the big time. Qeexo expects Impact Force to make an appearance on a smartphone soon. The company hinted that it may happen after the summer. For Touch Tools, Qeexo is working with an OEM partner, that Sang wouldn’t name for obvious business reasons, and is hoping to have it inside a device we can buy before the end of 2016.

Notice we said, ‘device’ rather than a phone or tablet? According to Sang, “It’s bigger than a tablet. Not a 55-inch TV or something, but between a typical tablet and TV. It will be targeting both regular people and businesses.”

Once these devices reach consumers, the times of ignoring gesture controls because we can’t remember what to do may be close to an end.

Mobile

Ironically, tech will save us from the horror of Cell Phone Courtesy Month

July is Cell Phone Courtesy Month, and while we could probably all do with being a little more considerate with our phone use, advancements in tech have the potential to end us staring at a little screen when we shouldn’t.
Photography

18 Lightroom keyboard shortcuts you can actually remember

Keyboard shortcuts make quick work of sorting and even editing photos, but they're often tough to remember. Here are 18 of the most helpful Lightroom shortcuts, whether you intend to use them in MacOS or on your Windows-based PC.
Computing

Microsoft’s pressure-sensitive software keyboard is ripe for dual-screen laptops

Microsoft is working on a new keyboard that will make it easier to type on glass. The software keyboard will be able to recognize pressure and gestures to help speed up your typing, especially when you're entering symbols.
Wearables

Apple patent hints at better gesture control on the Apple Watch

The Apple Watch already does things like shows you the time when you raise your wrist, but soon the gesture controls on the Apple Watch may get a whole lot better, if Apple's latest patent is anything to go by.
Computing

Microsoft patent paves way for a touch- and smudge-free tablet future

While touchscreens are great for navigating on smartphones and tablets, your greasy fingers leave annoying prints on the surface, blurring the view. Microsoft aims to change that using a design that relies on touch-free input.
Product Review

Putting on the grits with the Instant Pot pressure cooker

If you want to dip your toe into the low-temperature waters of sous vide, the Anova precision cooker is a good way to start. It has a robust app and an easy-to-use interface that just may convert you to a whole new way of cooking.
Computing

You can use Chrome OS on a tablet, but it’s not an iPad competitor yet

We took a look at the first Chrome OS tablet to hit the market, the Acer Chromebook Tab 10, to see just how Google's lightweight tablet runs on a touch-only device thanks to a few new tablet-centric features.
Photography

6 pro tips for taking amazing portrait photos with the Huawei P10

To really show us what the P10's Leica camera and its Portrait mode can do, a pro-photographer taught us how to use it. We learned plenty. Here are the best tips for perfect portraits and selfies.
Computing

Lost without Print Screen? Here's a few ways to take a screenshot on your Mac

Whether you prefer to use keyboard shortcuts or applications such as Grab and Preview, this guide will teach you how to take a screenshot on a Mac. Once you know how, you'll be able to capture images within seconds.
Computing

Calibrate your display to get it looking just how you like it

Want to see images the way they're intended to be seen? Here is our quick guide on how to calibrate your monitor using your operating system or another tool, to make what's on the screen look as good as it can.
Computing

Microsoft calls for Congress to regulate use of facial-recognition tech

Although highly beneficial, facial-recognition technology could also be exploited for use in malicious ways. Now, Microsoft wants Congress to intervene to create laws to help govern the use of the technology in society.
Computing

Congressman says we should be banned from mining, using cryptocurrency

Congressman Brad Sherman believes the government should prohibit U.S. citizens from mining and using cryptocurrency. As a medium of exchange, cryptocurrencies facilitate narcotics trafficking, terrorism, and tax evasion.
Wearables

Look Ma, no hands! The Horizon watch discards tradition for daring simplicity

Think a watch always has to have hands, or at least a way to tell the precise time? Think again. The Optik Instruments Horizon watch doesn't have any hands, but still manages to provide an at-a-glance time that's precise enough for most of…
Computing

Gaming on a laptop has never been better. These are your best options

Gaming desktops are powerful, but they tie you down to your desk. For those of us who prefer a more mobile experience, here are the best gaming laptops on the market, ranging from budget machines to maxed-out, wallet-emptying PCs.
Emerging Tech

Scientists gave gerbils a futuristic ear implant that lets them hear light

Researchers at Germany’s University Medical Center Göttingen have demonstrated a technique that involves using flashes of light to restore hearing in gerbils. Here's how the tech works.
Cars

Uber may soon turn into your personalized mini convenience store

Taking an Uber may soon be an all-inclusive experience. The transportation giant recently partnered with Cargo, the startup that offers in-car shopping experiences to those on the move.
Product Review

The Moto Z3 Play packs a lot of juice with its included battery mod

Motorola’s latest phone is the mid-range Moto Z3 Play. It costs $500, so why wouldn’t you just get the OnePlus 6? The answer isn’t so simple. There’s a lot to like here, and the phone comes with a battery mod that extends its life…
Mobile

Best iOS app deals of the day! 6 paid iPhone apps are free for a limited time

Everyone likes apps, but sometimes the best ones are a bit expensive. Developers put paid apps on sale for a limited time, but you have to snatch them up fast. Here are the latest and greatest iOS app deals offered in the iOS App Store.
Mobile

Here’s how to unlock your phone automatically with Android Smart Lock

Tired of unlocking your smartphone with a PIN or passcode? Android Smart Lock unlocks it automatically using your location, face, and more. Here's how to set it up, and everything you need to know about it.
Mobile

FCC begins accepting petitions to deny the T-Mobile and Sprint merger

T-Mobile and Sprint are hoping the third time is the charm for a merger. It comes just months after the carriers cited an inability to find common ground, extending yet again the long dance between the two telecom giants.
Mobile

Huawei may use a perforated LCD display to create a bezel-less Huawei Mate 20

Huawei has apparently put in an order for massive, 6.9-inch OLED screens, which will be used on a new smartphone coming later this year. Potentially its 2018 Mate series phone, it will challenge other big-screen devices we're expecting.
Home Theater

Set your ears free with the best completely wireless earbuds

If you can't stand the tangle of cords, or you're just excited about completely wireless earbuds, you're going to need some help separating the wheat from the chaff. Our list serves up the best wireless earbuds around.
Mobile

Some Samsung phones aren’t alerting users of app background processes

Worried that Facebook is spying on you? Some Samsung phones aren't correctly showing some apps on the App Permission Monitor, so users won't be alerted if an app tries to access certain permissions.