Skip to main content

Microsoft patent paves way for a touch- and smudge-free tablet future

Image used with permission by copyright holder

While touchscreens are great for navigating on smartphones and tablets, your greasy fingers leave annoying prints on the surface, blurring the view. Microsoft aims to change that, at least on future Surface devices, by implementing a design that relies on touch-free input. This design appears in a granted patent published by the U.S. Patent and Trademark Office on July 10 simply named as “Touchless input.”

Similar to how Apple’s TrueDepth camera on the iPhone X maps your face to eliminate the need for login credentials, Microsoft’s design would rely on a similar camera that captures a series of depth maps of an entire scene, including a human subject. A detection system built within the device scans each pixel in each map to determine what is static junk and what consists of the device owner.

According to the patent, this process includes “modeling the human subject with a virtual skeleton including a plurality of virtual joints including a virtual hand joint and one or more other virtual joints.” These virtual joints are compared across all captured depth maps to determine a finger gesture based on machine learning. The system can track more than one human user and/or physical objects.

Image used with permission by copyright holder

In an example, as the device owner is navigating the user interface, the integrated tracking system is tracking the position of the user’s finger in physical space. The device maps the physical location of the finger to the on-screen cursor. That said, the user can simply move the finger in physical space to move the cursor in the same direction without having to actually touch the screen’s surface.

But the tracking system doesn’t just support simple cursor movements. Given that it can track multiple fingers, users can click and drag files and folders, zoom in and out, pan across the screen, double-tap, rotate, and more. To that extent, Microsoft is emulating mouse functions and physical gestures used on touchscreens and trackpads, only they are performed in the air instead of on a hard surface.

“In embodiments in which the depth camera has sufficient resolution, fingerprints of a human subject may be identified and matched to a user,” the patent states. “In this way, the user can be identified, and cursor control or other computing attributes may be set to preferences associated with the identified user.”

Device configurations may include an additional visible-light camera that may or may not have the same resolution as the depth camera. It would be used to detect the light intensity of a surface using red, green, and blue values, thus adding color to the depth maps to provide more detail. Another configuration would use a “structured light” depth camera that projects a structured infrared illumination.

Microsoft notes that the touch-free control would be integrated into the operating system as a natural user input, joining your typical mouse, keyboard, touchscreen, and game controller. Processing the actual finger gestures via machine vision and/or gesture recognition may be handled on- or off-board, depending on the setup.

Editors' Recommendations

Kevin Parrish
Former Digital Trends Contributor
Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then…
How to cancel a Twitch subscription on desktop or mobile
The Twitch desktop app.

There comes a time in every online gamer’s life when one must move on from platforms once cherished. Beyond PlayStation Plus and Xbox Game Pass, one of the premiere gaming content meccas is Twitch. Built from the ground up to give players the world over a community forum to live-stream through, it’s the kind of content hub that’s great when you want it, but maybe one of the first things you want to walk away from when you’re taking a break from your PC and consoles.

Read more
How I unlocked the hidden modes of DLSS
dlss hidden modes dt respec

Nvidia's Deep Learning Super Sampling (DLSS) has become a mainstay in modern PC games. We all know about the basic presets to choose from in games that set the quality level and tip the scales toward performance or image quality.

But under the surface, there are a range of hidden presets that make DLSS behave in different ways. These are how developers tweak how DLSS reacts to a given input resolution and specific game content. They aren't meant to be user-facing, but a clever mod allowed me to open up the hood of DLSS and get my hands dirty. Not only do these hidden presets provide far more customization,  but they also reveal how DLSS really works.
Meet DLSSTweaks

Read more
The 6 best tablets for college in 2024
Using the Apple iPad Air 5.

If you pick the right tablet, there's a good chance it'll last you all four years of college –possibly even into grad school or your first job. However, narrowing down all the options can be a challenge. Not only do you need something reliable and portable, but you'll want it to support all the programs you'll be using on a daily basis. And unless you want to be staring at a muddled screen, it should also come with a vibrant screen and high-quality resolution.

Before starting your search for a tablet for college, you'll want to figure out how you want to use the device. Will you be taking notes on it? Attending classes remotely? And what sort of software do you need it to run? Once you've answered those questions, you'll then have to set a budget, as tablets run the gamut from affordable to ultra-expensive.

Read more