Skip to main content

MIT researchers develop eye-tracking program for your smartphone

researchers mit georgia eye tracking program app itracker3
Visualhunt
According to a new paper, researchers at the University of Georgia and the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory have developed a piece of software capable of turning almost any smartphone into an eye-tracking device.

Inspired by what MIT graduate student Aditya Khosla calls a “chicken-and-egg loop,” regarding eye-tracking applications for smartphones, researchers teamed up to bring together the information needed to develop such a program. In speaking to MIT News, Khosla said:

“Since few people have the external devices, there’s no big incentive to develop applications for them. Since there are no applications, there’s no incentive for people to buy the devices. We thought we should break this circle and try to make an eye tracker that works on a single mobile device, using just your front-facing camera.”

In order to create such a program, the team of engineers and researchers first needed data, the key ingredient in creating the brain of the program, a machine learning algorithm.

To do this, the researchers sought out strength in numbers with a little crowdsourcing from Amazon’s Mechanical Turk program. Previous research in this area was limited to only 50 or so subjects, who were asked to come into a lab to complete the tests.

A collection of the subjects who participated in the data collection stage of development
A collection of the subjects who participated in the data collection stage of development Image used with permission by copyright holder

Khosla and his team far exceeded previous research by sampling the eye movements of 1,500 mobile users, all of whom participated with the help of GazeCapture, a custom iOS application that combined on-screen animations with use of the front-facing camera.

Diagram showing how the data is used to predict eye movement in subjects
Diagram showing how the data is used to predict eye movement in subjects Image used with permission by copyright holder

The data captured from 800 of the 1,500 subjects was used to create iTracker, the first iteration of the eye-tracking system, which had a margin of error of only 1.5 centimeters. Since then, the researchers have analyzed the remaining 700 subjects and further improved the margin of error, shrinking it down to just 1 centimeter.

The team is hopeful that further testing will be able to reduce that even more, down to a margin of error of a half-centimeter, the key number that will make the program commercially viable in the eyes of Khosla.

As for the practical applications for such technology, it can be used for almost anything more conventional eye-tracking methods are used for, from helping doctor’s discover and keep tabs on various illnesses, to market research and analytics. The key difference here is that this new method of tracking is far less invasive and exorbitantly less expensive than alternative options, making it easier than ever to capture larger quantities of data.

Below is the paper in its entirety:

It’ll be interesting to see what the future holds. The team behind it will be presenting the paper on June 28 at the Computer Vision and Pattern Recognition conference in Las Vegas.

Editors' Recommendations

Gannon Burgett
Here are Apple’s secret plans for adding AI to your iPhone
A person holding the Apple iPhone 15 Plus and Apple iPhone 15 Pro Max.

Apple iPhone 15 Pro Max (left) and Apple iPhone 15 Plus Andy Boxall / Digital Trends

After the AI gala that was Samsung’s Galaxy S24 series phones, Apple could be the next to tap into the magic of deep learning and Large Language Models (LLM) that power tools such as ChatGPT and Google Bard. According to an industry analysis report by Financial Times, Apple has been on a hot acquisition streak, team reorganizations, and fresh hiring to develop AI capabilities for iPhones.

Read more
Check your iPhone right now for an important security update
iOS 17 interactive widgets on an iPhone 15 Pro Max.

If you're someone who uses an iPhone, you'll want to pay attention to a new software update Apple just released today. Why? It adds an important new security feature to your iPhone.

The update in question is iOS 17.3, and the hallmark feature it adds is a new tool called "Stolen Device Protection." Announced in December, Stolen Device Protection makes it harder for thieves to access your iPhone if it's lost or stolen. As Apple describes it, "this new feature adds an additional layer of security in the unlikely case that someone has stolen your phone and also obtained your passcode."

Read more
Look who just replaced Samsung as king of the global smartphone market
The Apple iPhone 15 Pro Max and the Samsung Galaxy S23 Ultra's cameras.

Samsung has been sitting pretty at the top of the global smartphone market for more than a decade. Until now, that is.

Data released this week by research firm IDC shows that the Korean tech giant has been knocked off its perch by archrival Apple thanks in part to robust sales of the iPhone.

Read more