Skip to main content

eyeSight wants to bring gesture control to smartphone-based VR

Virtual reality and augmented reality are both on the rise, but companies are still figuring out the best way to input commands for altered reality devices. Devices like the Oculus Rift rely on more traditional controllers, but as time goes on there are sure to be better ways to input information. Like with our hands.

eyeSight Technologies’ new tech allows you to input commands into augmented or virtual reality devices without the need for extra hardware. In other words, with this tech, you can control even smartphone-based augmented and virtual reality.

Recommended Videos

“As the era of virtual reality emerged the need for natural touch-free control over virtual content became ever so clear, as no buttons or controllers can provide a truly immersive virtual experience like control using the hands,” Gideom Shmuel, CEO of eyeSight Technologies, told Digital Trends. “Allowing users to interact with virtual content by simply pointing a finger elevates the entire experience with VR, and this is something we seek to do on a mass market scale.”

The tech uses the rear-facing camera of both Android and iOS phones, through which it can detect motion. By using this motion, your hands can control different aspects of the interface in virtual and augmented reality. In the example given we can see a game being controlled in augmented reality, however it could also be used in virtual reality — while you wouldn’t see the image being captured through the camera, that doesn’t mean that image data can’t be analyzed.

eyeSight Gesture Control for Smartphone-Powered VR

Of course, all this is still in its early stages, and it sounds like a pretty processor-hungry technology, especially considering the fact that smartphone-based augmented and virtual reality apps are already a big strain on a smartphone. Add in camera use, plus the fact that images have to be analyzed in real-time, and that smartphone has a lot to take care of. Still, that doesn’t mean the technology isn’t cool. As smartphones get better, virtual reality and augmented reality on smartphones will also get better, and eyeSight’s new tech could open the door to a range of cool new apps.

“Most VR and AR applications require user interaction, and our aim is to truly transform the experience into an immersive one with touch-free control,” continued Shmuel. “We seek to bring our computer vision technology to applications requiring human interaction such virtual conferencing for example where participants in a virtual meeting can point at screens and documents, educational purposes like navigating through virtual medical procedures, or even pointing at a painting in a virtual museum to learn about its features.”

To date there haven’t really been many gesture-controlled smartphone VR apps, although a quick search does turn up a gesture-controlled VR video player. It will be interesting to see how gesture control can change smartphone-based virtual reality.

Updated on 05-17-2016 by Christian de Looper: Added comments from eyeSight CEO Gideom Shmuel.

Christian de Looper
Christian de Looper is a long-time freelance writer who has covered every facet of the consumer tech and electric vehicle…
This one iPadOS 26 feature has me excited for the iPhone Fold
Semi-open state of a foldable iPhone concept

Samsung is set to launch the seventh generation of its Galaxy Z Fold book-style folding phone this Summer, but its biggest rival is yet to show its folding phone hand. Apple has long been expected to unveil an iPhone Fold, and the latest rumors suggest that it will launch next year.

I’ve used almost every folding phone released globally, with some exceptions for extremely obscure ones. While I've always been curious what an iPhone Fold would look like, I was fairly certain that Apple shouldn't build it, as I wasn’t sure they could deliver on one necessary feature.

Read more
These are the 4 best alternatives to the Google Pixel 9a
A person taking the Google Pixel 9a out of a pocket.

The best cheap phone race has been reignited by the Google Pixel 9a, which brings the Google Pixel 9 series experience at a significantly lower price. Starting at $449, Google’s latest Pixel offers the flagship processor of its siblings for flagship-level performance, an excellent camera, and long-term software support.

Yet, Google’s software experience won’t be for everyone; there are obvious tradeoffs in the specs sheet to enable the low price; the camera bar design is somewhat polarizing for long-term fans, and there’s no telephoto camera. 

Read more
Apple and Samsung rivalling phones, earbuds and more due at huge July event
The OnePlus Summer Launch Event has five new products, including the Nord 5 and Pad 3 Lite
Five OnePlus products launching in July 2025

OnePlus has confirmed it will host a bumper launch event on July 8, where it will announce five new products, covering phones, headphones, wearables and tablets.

The range of new devices will join the OnePlus 13, OnePlus 13R and OnePlus Pad 3 which arrived earlier this year as the Chinese firm fills out its product offering to compete against Apple, Samsung and co.

Read more