Researchers at Microsoft Research have developed a way to give HoloLens users a virtual touchscreen through a system called MRTouch. This allows users of Microsoft’s mixed reality headset an additional way to interact while using HoloLens, complementing gesture, voice, and controller inputs, according to a Microsoft Research video demoing the use of the MRTouch prototype.
Although Microsoft Research hasn’t announced plans to bring MRTouch to the market or allow third-party developers to make use of the multi-touch interactions at this time, the good news is that it works on an unmodified Microsoft HoloLens headset. All users would need to do is use their fingers and swipe across a flat surface to create a virtual touchscreen. You can do this on a number of surfaces, including walls and tabletops. This virtual touch area could be used to display content, and you can interact with the virtual screen using multi-touch gestures, similar to how you would use a tablet.
One application for MRTouch and HoloLens is the ability to launch apps that rely on touch inputs, like a browser. This allows you to use a mixed reality device as your day-to-day computer, Microsoft said. In the photos app, Microsoft showed that you can pan and zoom using your fingers, just as you would on the Windows 10 Photos app on your touchscreen-enabled laptop or convertible. MRTouch even edges ahead of traditional touchscreens if you need to render realistic 3D content. Here, touch input could be combined with in-air gestures for specific interactions in three-dimensional space.
MRTouch essentially allows HoloLens owners to overlay a touchscreen that’s capable of displaying and rendering content on any flat surface.
MRTouch works by utilizing the short-length depth camera on the HoloLens headset for finger tracking. A reflectivity map is also captured at 25 frames per second, and the data is fed to the tracker engine, Microsoft said. Microsoft’s study tested for accuracy, and the results show that touch was detected 97.5 percent of the time with a mean distance error of 5.4mm. According to Microsoft, the accuracy rate of MRTouch is very competitive with the capacitive touchscreens today used on many modern laptops, tablets, and smartphones.
