HTC Vive Pro Cameras Can be Used to Track Hands, Controllers

HTC Vive Pro cameras can be used to track hands, controllers

Beyond the uprated screen resolution over its predecessor, the HTC Vive Pro’s most obvious change is in the form of a secondary external camera on the front. Although speculation suggested that the stereoscopic array could enable augmented reality, it turns out that the cameras are used for depth sensing, which can improve the safety-orientated chaperone feature and enable basic controller and hand tracking.

One of the major difficulties faced by virtual reality companies in headset and controller development is occlusion. If something gets in the way of the sensors, suddenly the system powering the headset doesn’t know where it is and that could get uncomfortable for the wearer. The HTC Vive’s Valve-developed Lighthouse trackers had less trouble with that than the Oculus Rift’s cameras, but it still wasn’t perfect.

HTC/Valve’s Lighthouse 2.0 tracking solution, due to release later in 2018, will improve this with support for four tracking stations, but the Vive Pro’s cameras could provide a secondary layer of anti-occlusion technology too. In an interview with HTC Vice President of VR technology Raymond Pao, Engadget learned that the stereoscopic cameras can track hands independently of controllers or Vive trackers.

That wasn’t its original intention, however. HTC’s initial plan was to use them to augment its chaperone feature which provides a digital wall to prevent VR users from hurting themselves and damaging hardware. The cameras would be able to provide a more nuanced chaperone tracking system, thereby showing objects which could be knocked over, or potentially pets or children getting in the way.

That is because the cameras are only capable of “low VGA” resolution (640 x 480), so wouldn’t be fun to look through for augmented reality purposes. However, because of their depth-sensing capabilities, early developers playing around with the Vive Pro found that they could provide basic hand-tracking. Although simple, it was enough to provide precise tracking for fingers, making it possible to play VR games without motion controllers.

That form of “inside-out” tracking does have its limitations — putting your hands outside of the view of the camera would break tracking — but it shows interesting potential. Even then it could be used as a backup tracking solution for controllers should the wearer’s body or another object block the line of sight to the mounted Lighthouse sensors.

Editors' Recommendations