eyeSight wants to bring gesture control to smartphone-based VR

eyeSight Technologies’ new tech allows you to input commands into augmented or virtual reality devices without the need for extra hardware. In other words, with this tech, you can control even smartphone-based augmented and virtual reality.

“As the era of virtual reality emerged the need for natural touch-free control over virtual content became ever so clear, as no buttons or controllers can provide a truly immersive virtual experience like control using the hands,” Gideom Shmuel, CEO of eyeSight Technologies, told Digital Trends. “Allowing users to interact with virtual content by simply pointing a finger elevates the entire experience with VR, and this is something we seek to do on a mass market scale.”

The tech uses the rear-facing camera of both Android and iOS phones, through which it can detect motion. By using this motion, your hands can control different aspects of the interface in virtual and augmented reality. In the example given we can see a game being controlled in augmented reality, however it could also be used in virtual reality — while you wouldn’t see the image being captured through the camera, that doesn’t mean that image data can’t be analyzed.

Of course, all this is still in its early stages, and it sounds like a pretty processor-hungry technology, especially considering the fact that smartphone-based augmented and virtual reality apps are already a big strain on a smartphone. Add in camera use, plus the fact that images have to be analyzed in real-time, and that smartphone has a lot to take care of. Still, that doesn’t mean the technology isn’t cool. As smartphones get better, virtual reality and augmented reality on smartphones will also get better, and eyeSight’s new tech could open the door to a range of cool new apps.

“Most VR and AR applications require user interaction, and our aim is to truly transform the experience into an immersive one with touch-free control,” continued Shmuel. “We seek to bring our computer vision technology to applications requiring human interaction such virtual conferencing for example where participants in a virtual meeting can point at screens and documents, educational purposes like navigating through virtual medical procedures, or even pointing at a painting in a virtual museum to learn about its features.”

To date there haven’t really been many gesture-controlled smartphone VR apps, although a quick search does turn up a gesture-controlled VR video player. It will be interesting to see how gesture control can change smartphone-based virtual reality.

Updated on 05-17-2016 by Christian de Looper: Added comments from eyeSight CEO Gideom Shmuel.