Tobii, the company behind the popular eye-tracking hardware for PC gaming, said on Thursday, March 15, that it teamed up with Qualcomm to bring eye-tracking capabilities to mobile headsets supporting virtual reality and augmented reality. The technology will be part of Qualcomm’s next-generation reference design and development kit based on its new Snapdragon 845 processor, which was first used in the Samsung Galaxy S9 phone launched in February.
Qualcomm’s head-mounted display (HMD) reference design will include Tobii’s proprietary EyeCore eye-tracking algorithms and hardware design. In turn, Tobii said it would work closely with HMD manufacturers to provide the best optical solution for their design. The company also plans to license its eye-tracking technologies and system to these manufacturers as well.
On a PC, Tobii’s eye-tracking hardware does just that — it tracks where the gamer is looking at all times to enhance the experience. For example, Assassin’s Creed Syndicate will automatically pan the worldview according to the gamer’s focus versus the player moving the mouse to pan the camera. Likewise, the system will automatically select a target in-game when the player views an enemy, or scanning places to attach a grappling hook.
On a Windows 10 PC, Tobii’s system can make navigation a bit easier. On a laptop, you can simply look at a specific point on the screen, and the cursor will jump to that point when placing a finger on the touchpad. Other capabilities in Windows 10 when using Tobii’s hardware includes managing apps using task view, the application switcher, snap assist, and more.
But with virtual and augmented reality scenarios, eye tracking can relieve a large chunk of the rendering overhead needed to create the experience. Right now, headsets render the entire environment crystal clear no matter your focal point. But with eye tracking, the overall performance increases by not fully rendering the scene outside your field of view. Eye tracking can enable this method, called “foveated rendering.”
Yet, that is not all. According to Tobii, eye tracking can enable HMDs to orient images based on your pupils. That indicates you will no longer need to manually adjust the lenses given Tobii’s hardware will adjust the scene for each individual, increasing visual quality. Another benefit is hand-eye-coordination: You won’t need to fully move your head like a robot to interact with virtual objects controlled by your hands.
“Devices can accurately track your gaze in real time, enabling content creators to express one of the most fundamental dimensions of human interaction — eye contact,” the company says. “VR technologies hold the promise of enabling a new and immersive medium for social interaction. The addition of true eye contact to virtual reality helps deliver that promise.”
Qualcomm’s Snapdragon 845-based reference design will be its second stand-alone HMD kit to date. The first kit, introduced in 2017, relies on the older Snapdragon 835 chip along with a single AMOLED screen with a 2,560 x 1,440 resolution. The recent stand-alone Google Daydream headsets are mostly based on this design along with HTC’s Vive Focus headset sold in China.
