Microsoft patent paves way for a touch- and smudge-free tablet future

new microsoft patent eliminates smudges with touchless input

While touchscreens are great for navigating on smartphones and tablets, your greasy fingers leave annoying prints on the surface, blurring the view. Microsoft aims to change that, at least on future Surface devices, by implementing a design that relies on touch-free input. This design appears in a granted patent published by the U.S. Patent and Trademark Office on July 10 simply named as “Touchless input.”

Similar to how Apple’s TrueDepth camera on the iPhone X maps your face to eliminate the need for login credentials, Microsoft’s design would rely on a similar camera that captures a series of depth maps of an entire scene, including a human subject. A detection system built within the device scans each pixel in each map to determine what is static junk and what consists of the device owner.

According to the patent, this process includes “modeling the human subject with a virtual skeleton including a plurality of virtual joints including a virtual hand joint and one or more other virtual joints.” These virtual joints are compared across all captured depth maps to determine a finger gesture based on machine learning. The system can track more than one human user and/or physical objects.

new microsoft patent eliminates smudges with touchless input 2

In an example, as the device owner is navigating the user interface, the integrated tracking system is tracking the position of the user’s finger in physical space. The device maps the physical location of the finger to the on-screen cursor. That said, the user can simply move the finger in physical space to move the cursor in the same direction without having to actually touch the screen’s surface.

But the tracking system doesn’t just support simple cursor movements. Given that it can track multiple fingers, users can click and drag files and folders, zoom in and out, pan across the screen, double-tap, rotate, and more. To that extent, Microsoft is emulating mouse functions and physical gestures used on touchscreens and trackpads, only they are performed in the air instead of on a hard surface.

“In embodiments in which the depth camera has sufficient resolution, fingerprints of a human subject may be identified and matched to a user,” the patent states. “In this way, the user can be identified, and cursor control or other computing attributes may be set to preferences associated with the identified user.”

Device configurations may include an additional visible-light camera that may or may not have the same resolution as the depth camera. It would be used to detect the light intensity of a surface using red, green, and blue values, thus adding color to the depth maps to provide more detail. Another configuration would use a “structured light” depth camera that projects a structured infrared illumination.

Microsoft notes that the touch-free control would be integrated into the operating system as a natural user input, joining your typical mouse, keyboard, touchscreen, and game controller. Processing the actual finger gestures via machine vision and/or gesture recognition may be handled on- or off-board, depending on the setup.

Product Review

Controversy has dogged the MacBook Pro lately. Is it still a good purchase?

The MacBook Pro is a controversial laptop these days -- and that's unfortunate. Due to some divisive changes Apple made to the functionality of the MacBook Pro, fans are more split. Does the 8th-gen refresh change that?

Detangle your desk with these mighty wireless mice

If you're looking for the best wireless mouse on the market, we've got the list for you!. Here are six models that will give everyone what they need, whether they're hardcore gamers or looking to ward off carpal tunnel.

Tap Strap wearable keyboard gains support for VR applications

TAP System's wearable keyboard gains support for virtual reality, now compatible with Windows Mixed Reality, Oculus Rift, and HTV headsets. Type and tap for up to eight hours in VR without needing to look at a physical keyboard.

Here's the Samsung Galaxy S9's new Android 9.0 Pie interface

The Samsung Galaxy S9 and Galaxy S9 Plus are here. The flagship devices boast some awesome new features and a powerful new processor. Here's everything you need to know about these Samsung phones.

What to expect from Microsoft’s October 2 Surface event

The October 2 Surface event is just around the corner. Let's take a look at everything Microsoft could unveil at the event, including major announcement for new Surface products, unique accessories, and long-sought Windows 10 features.

MacOS Mojave has landed. Here are the five best features of the update

Apple's new version of MacOS is finally here and can be installed on your Mac today. Why should you upgrade? Well, there's a lot more to the update than just Dark Mode. Here are the five features you need to know about.

MacOS Mojave launches on September 24. Here's what we like about it so far

Mojave is the latest version of MacOS, and it's out now. Chock-full of quality-of-life upgrades, we took it for a test drive to get a sneak peek at what you can expect from the next major update to MacOS.

How to enable dark mode in MacOS Mojave

Learn how to enable dark mode in MacOS Mojave! As Mac's latest update gears up to hit all Mac systems later this year, the public beta for Mojave is nearly open for you to test out the latest features. One of the most in-demand changes is…

Critical MacOS Mojave vulnerability bypasses system security

Security Researcher Patrick Wardle has discovered a critical MacOS Mojave security flaw that could potentially allow malicious applications to bypass Mac's system security controls.
Emerging Tech

Microsoft and Shell build A.I. into gas stations to help spot smokers

Shell and Microsoft have created a system for gas stations that can spot someone who's smoking or about to smoke. The platform uses multiple cameras, local computing power, and Microsoft's cloud intelligence system to do the job.

If your data is found on the dark web, Firefox Monitor will let you know

Firefox is finally launching its Firefox Monitor service and you don't have to use the Firefox browser to access it. Monitor scans the dark web to see if your email address has been leaked as part of a past data breach.

Chrome OS update could make switching to tablet mode far easier

Google is working on an update for Chrome OS that would make its browser-based operating system much easier to operate in tablet mode, even with the new, streamlined user interface.
Emerging Tech

Teaching machines to see illusions may help computer vision get smarter

Researchers are teaching computers to see optical illusions. The reason? To create smarter, more brain-like vision recognition algorithms for everything from robots to autonomous cars.

How many GPU video ports is too many? The Aorus RTX 2080 packs seven

Aorus' new RTX 2080 graphics card wants to turn up the new-generation GPUs to 11 with greater cooling, RGB lighting, and a whole host of video port options that give anyone more than they'll likely ever need.