Skip to main content

Intel’s incredible athlete-tracking A.I. is the ‘holy grail’ of training tech

Intel and EXOS Pilot 3D Athlete Tracking with Pro Football Hopefuls

Jonathan Lee, director of sports performance technology in Intel’s Olympic Technology Group, has flown 11 hours and quarantined for 14 days to stitch skeletons together on a computer. And if it works as well as he hopes, it’ll be an amazing innovation for action replays at the 2020 Toyko Olympics.

“Part of our artificial intelligence is [designed] to stitch together the right skeletons when you have up to eight or nine athletes running down the track,” Lee told Digital Trends from his hotel room in Tokyo’s Olympic Village, 5,000 miles from his home in San Francisco.

He is part of a crack team of Intel engineers that have been dispatched to the Tokyo Games to incorporate Intel’s 3D Athlete Tracking (or 3DAT, pronounced “three-dat”) technology into this year’s Olympic broadcasts. 3DAT’s breathtaking overlay visualizations will be made available during replays of the 100-meter, 200-meter, 4×100-meter relay, and hurdle athletic events, taking place between July 30 and August 4.

Crack team of Intel engineers setting up 3DAT equipment at Olympic Trials.

“3D Athlete Tracking … is a technology we have developed here at Intel that allows us to take standard video of athletes and extract information about their form and motion,” said Lee. “We do this using A.I. and computer vision. [Using our technology, we can] recognize the different parts of the body, from the eyes and nose all the way down to the ankles and toes, and use this to construct a 3D skeleton of the athlete or, in some cases, multiple athletes. From those skeletons, we can then extract information like velocity, acceleration, and biomechanics.”

To put it simply, 3DAT fuses together video taken from multiple 4K machine vision cameras with broadcast footage and uses this to create a three-dimensional model of Olympians in action. These can be used to provide computer-generated replays of the action. What gives this the edge over traditional video replays, however, is that the 3DAT technology can ingest its various sources of video footage and use this to generate motion-capture models that can be rotated in 3D space.

“You give the [broadcaster] the ability to, in essence, rotate, zoom, put the camera wherever they want to,” said Lee.

Heat maps and beyond

It’s not just rotating the “camera,” either. By extracting data like velocity and acceleration from the 3D models, 3DAT can overlay the models with added information such as heat maps to indicate how fast an athlete is running, when they hit their top speed, and how long they’re able to maintain this speed. It’s a level of eye-catching data visualization never before tried at the Olympic Games — or just about anywhere else, for that matter.

Sha'Carri Richardson makes a STATEMENT with dominant 100m heat at trials | NBC Sports
“What you want is something that is useful and beautiful and helps the viewer at home really connect up with the athletes and understand something that they didn’t know before,” said Lee.

Motion capture is, of course, nothing new. It’s been used in Hollywood for years, most notably in some of the astonishing theatrical performances captured by companies like Weta Digital from actors like Andy Serkis (who has played everyone from Gollum in The Lord of the Rings to Caesar in Dawn of the Planet of the Apes to King Kong in, err, King Kong). Mo-cap is also frequently used in the gaming world to ensure that on-screen avatars move as closely as possible to real people. But while many mo-cap suits are studded with sensors to capture the movement of individual limbs, 3DAT requires zero sensors.

The problem, Lee said, is that while motion capture suits are fine for certain scenarios, tracking elite-level athletes is not necessarily among them.

“Imagine you put a sensor on someone’s head, elbow, chest, and then you say, ‘All right, go do a high jump, and when you land, you’re gonna feel all these sensors being pressed into your body,’ right?” he said. “You can imagine that’s not necessarily a pleasant experience. Or [how about] a sprinter? If the sensor is placed right below the knee, that’s going to interfere with how they come out of the blocks and how they run.”

Instead, 3DAT relies entirely on computer vision and pose estimation algorithms to analyze the biomechanics of athletes’ movements. Lee said that this can be done accurately enough to capture even the smallest nuances of an athlete’s motion. No trackers required.

The future of 3DAT

Crack team of Intel engineers siting in front of monitors using 3DAT at Olympic Trials.

With this in mind, Lee doesn’t just see 3DAT as a data-viz tool for entertaining and informing the viewers at home. It’s also being used as a training tool for athletes to review their performance. “We’ve had three different elite-level coaches use the exact same phrase to describe this as the ‘holy grail of coaching,’” he said. One potentially transformative use case? Helping to diagnose injuries.

“One of the biggest things that affect athletes are soft tissue injuries,” Lee said. “These usually start to show up in the form of asymmetry. So perhaps your left and right stride length might become different, or the way that your hips move might [change], right? These are things that can happen even before an injury occurs. If you start to look at an athlete a little more longitudinally — say, at the beginning of the season, during, after, maybe during some sort of functional movement assessment — [an A.I.] can take over to recognize precursors for injuries, [so that the coaches and athletes] can ward them off before they happen.”

For this reason, Lee sees the future of 3DAT being increasingly wrapped up with artificial intelligence. “The question that comes up is what do you do with all this data?” he noted. “This is where I see the next frontier for 3DAT … There has to be a next level, whether that’s for injury prevention or for improving performance or helping rehabilitation … That next question is really the one that we’ll need to answer to be able to turn this from a technology that’s [really] cool into a technology that’s cool and [incredibly] useful.”

For now, though, Olympics viewers will have to settle for just “really cool.” Something tells us that will probably be enough. Coming soon to a gold medal event near you (or, at least, on your television).

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Smart home trends to look for in 2022
Astro Robot lifestyle image.

It's a brand-new year, and that means new and exciting innovations in the world of smart home technology. Although no one can say for sure what this year might hold for the ever-expanding industry, we can make a few educated guesses based on the direction it is currently headed, as well as with what we know from CES 2022 so far.
Autonomy and robots
Whether we're ready for it or not, it's clear that the biggest names in the smart home world are devoting a not-insignificant amount of time and research to household robots. While the idea of a semi-intelligent robot that can handle the cleaning around the house isn't unwelcome, it can also be a bit unsettling.

With that in mind, Amazon's Astro will likely see more widespread adoption, as will the Ring Always Home Cam. In 2020, Sunflower Labs launched The Bee, an autonomous home security drone for your yard. Samsung's JetBot AI+ might be a robot vacuum, but its built-in home security features make it closer to a home robot than many other devices on the market.

Read more
CES 2022 shows the very convoluted future of video game tech
Asus ROG Flow Z13 gaming laptop.

A lot has changed about the technology we use to play video games. For decades, players needed to have a PC, console, or portable device (whether it be a Game Boy or iPhone) to game. Nowadays, the lines between all three of those categories have blurred. Devices like the Nintendo Switch have sparked a portable console revolution, while cloud gaming is making companies rethink what devices gamers need to play AAA titles.

That philosophical shift has been on full display throughout CES 2022. Manufacturers showed up in full force to unveil incredibly powerful devices that further break the established gaming mold. While there are plenty to ooh and ahh at, but this year's show highlights just how convoluted gaming tech has become.

Read more
Digital Trends’ Tech For Change CES 2022 Awards
CES 2022 Tech for Change Awards Post Feature

CES is more than just a neon-drenched show-and-tell session for the world's biggest tech manufacturers. More and more, it’s also a place where companies showcase innovations that could truly make the world a better place -- and at CES 2022, this type of tech was on full display. We saw everything from water-conserving showers to affordable electric trucks. But of all the world-changing tech on display this year, these four innovations impressed us the most:

Whill Model F
No matter how advanced or capable they might be, most powered wheelchairs available today suffer from the same fundamental flaw: They aren’t easy to transport when they’re not in use.

Read more