Your wandering gaze controls the game with Sony’s futuristic eye-tracking tech

sonys research eye tracking video games absolutely amazing sony playstation magic labs

While Sony managed to earn more than a few impressed exclamations from press, fans, and members of the gaming community with its new virtual reality headset code named “Project Morpheus,” one of its more incredible pieces of hardware sat unnoticed in a back room and had nothing to do with virtual reality.

The tech in question is a piece of unnamed prototype hardware that may or may not ever make it to retail, but it should – it literally offers a new way to look at games. Using a specially designed camera, one of Sony’s research and development teams (the group known as Magic Labs) has come up with a way for you to control the camera in a game with your eyes. It isn’t something that changes the nature of gaming in the same way that virtual reality does, but it is one of the coolest pieces of gaming tech on display at GDC.

The eye tracking technology is still a long way from being released – and there is no guarantee it will ever be – but the potential is incredible. Eye tracking is becoming a common feature of smartphones, but the scale of Sony’s device eclipses anything a smartphone currently offers. In the demo on display, Sony’s eye tracking tools (which, again, don’t technically have a name at the moment) were used in conjunction with the PlayStation 4 game Infamous: Second Son.

To help the energy blasts reach their targets, you simply look at what you want to hit.

It all starts with a quick calibration that’s as simple as looking at one dot in the upper left corner then another on the diagonally opposite side. The camera then begins to send data to a nearby laptop, which in turn takes over for the game’s camera.

It takes a few moments to orient to the new method of seeing. In a third-person game, it’s natural for your eyes to gravitate to your character that is below the center of your screen. With the eye tracking software, looking at your character therefore causes the camera to pan down. Once you compensate, though, the effects are amazing.

Beyond the ability to look around with your eyes in the same way you would normally pan the camera around with the right thumbstick, the tracking can also be far more precise. In Infamous, quickly tapping the R2 button sends a stream of energy blasts at opponents similar to firing a machine gun, but with slower projectiles. To help the energy blasts reach their targets, you simply look at what you want to hit. If you are facing two enemies, one on either side of you, just glance one direction then the other and the energy blasts will find their mark. It registers your movements as fast as your eyes move, creating a responsive camera that could alter the way we visually input games forever.  

While the demo focused on what, for all intents and purposes, is a third-person shooter (at least in the demo, which showed an early section of the game where energy blasts were among your most effective weapons), the possibilities with Sony’s eye-tracking tech are endless. Imagine playing a soccer game and, instead of trying to awkwardly pass to one particular player running in an unusual pattern, you just look at them and hit the pass button. In football, while controlling the quarterback, you could simply look at your receivers and hit pass to the one you want. In first-person shooters, you could theoretically aim faster than your thumb could move.

There are also possible uses for the tech with the Project Morpheus virtual reality headset. It’s still just theoretical right now (as far as we know), but we imagine that you may be able to one day put on the VR headset and look around the virtual world by moving your head, but also look around as you naturally would, with your eyes.

sony eye tracking Magic Labs

 The effects could go beyond gaming as well. Imagine watching a 3D telecast of a basketball game using a virtual reality headset. You could look around a limited field of view, but when the action happened, your eyes would command the video where to focus. If a shot is blocked and a fast break begins, it’s more natural to follow with your eyes than whip your head around to see the ball race down court. 

There are some drawbacks, however. For one, it requires a new camera separate from the PlayStation 4’s camera. That said, the device on display was a pure prototype, and there could always be a chance they find a way to incorporate existing cameras to use the tech via software updates. But if it were released tomorrow, you’d need to buy a new peripheral.

Another, more significant, drawback is the software’s issues with tracking eyes behind glasses. It’s hit or miss on whether or not it recognizes the movement of the eyes behind lenses, and people who wear glasses will run into issues. That could eventually be fixed. But even if you are free of spectacles, you’ll need to sit close to the camera. In order to track the movement of eyes, a camera needs to be close enough to follow the movements, potentially creating an awkward issue with positioning. The alternative would be a high-resolution camera capable of zooming in enough to make out your eyes from a distance. That’s certainly possible, but it would likely cost more than it would be worth. Still, these are all solvable issues.

Sony’s eye-tracking tech was first shown back in November 2013 as the world prepared for the arrival of the PlayStation 4. It was highlighted as part of the look at Magic Labs, the company also responsible for devices like the Move, the Eye Toy Camera, and most recently the DualShock 4. At the time, the eye tracking was little more than a gimmick – a neat gimmick – but compared to the release of the new system, a theoretical prototype was not a major focus. Now that the PlayStation 4 is on the market, however, the boundaries of gaming are ready to be pushed by devices like this one, Project Morpheus, and others we haven’t considered yet (but are likely being worked on in labs around the world).

With the Project Morpheus casting a long shadow, eye tracking is still just a sideshow, especially since there are no firm plans for it at the moment. Give it time, though. This may just be part of the future of gaming.

Emerging Tech

Awesome Tech You Can’t Buy Yet: camera with A.I. director, robot arm assistant

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Health & Fitness

Still tired? Smart sleep aids can help improve your snooze

Do you snore? Toss and turn? Alternate between extremely hot and extremely cold body temperatures at night? A deluge of smart sleep technology on the market aims to help you get a better night of shut-eye.
Outdoors

Is Uber planning to put its self-driving tech into bikes and scooters?

Uber reportedly has its eye on building autonomous electric bikes and scooters that ride to a user when summoned by an app. The technology could also be used to make its two-wheelers safer with obstacle avoidance systems.
Gaming

Immerse yourself in a new universe with these incredible PSVR games

The PSVR has surpassed expectations and along with it comes an incredible catalog of games. There's plenty of amazing experiences to be had so we've put together a list of the best PSVR games available today.
Wearables

Omron HeartGuide brings blood pressure monitoring to your wrist

High blood pressure leads to heart attacks, strokes, and many other health problems, so it's important to keep an eye on. Omron's HeartGuide is a fitness tracking watch that can also monitor your blood pressure from your wrist.
Gaming

Xbox app lets you access your console while away from home. Here's how

Microsoft's Xbox allows you to access your profile information and launch media content directly from your mobile device. Check out our quick guide on how to connect your smartphone to an Xbox One.
Gaming

Goodbye, Machinima: YouTube gaming channel pulls the plug after 13 years

Machinima, a YouTube gaming channel that was launched in 2006, has suddenly shut down, with all of its videos set to private. The #RIPMachinima hashtag is now making the rounds in social media to reminisce about the channel's content.
Gaming

These are the best indie games you can get on PC right now

Though many indie games now come to consoles as well, there's still a much larger selection on PC. With that in mind, we've created a list of the best indie games for PC, with an emphasis on games that are only available on PC.
Gaming

This list of PlayStation 4 exclusives puts its competitors to shame

The PlayStation 4's game library and incredible selection of exclusive games could make anyone with an Xbox One or Nintendo Switch think twice. Here's our list of the latest and greatest PS4 exclusives.
Gaming

Overkill’s ‘The Walking Dead’ delayed indefinitely for PlayStation 4, Xbox One

Overkill's The Walking Dead for the PlayStation 4 and Xbox One have been postponed without a new release date. The co-op zombie shooter suffered massive delays, then received lukewarm reviews once it launched for the PC through Steam.
Gaming

The hottest Nintendo Switch games you can get right now

The Nintendo Switch's lineup started off small, but games have steadily released as the console continues through its second year. Here are the best Nintendo Switch games available now.
Gaming

How you can give your PS4 a fresh start with a factory reset

Learn the many ways you can factory reset your PS4. From reverting your settings to factory to doing a full wipe and reinstalling the latest PlayStation firmware, we cover it all here, step by step.
Gaming

Can't stand keyboard gaming on PC? Here's how to use a PS3 controller instead

Properly connecting a PlayStation 3 Controller to a PC is no easy task, especially when you opt for third-party peripherals. Thankfully, our guide will help you through the process.
Product Review

‘Resident Evil 2' is a terrifying new virus you'll want to catch

Resident Evil 2 brings the Raccoon City incident to a new generation of players, acting both as a nostalgic throwback and a thoroughly modern horror game with some of the best visuals of the generation.