Skip to main content

Intel’s incredible athlete-tracking A.I. is the ‘holy grail’ of training tech

Intel and EXOS Pilot 3D Athlete Tracking with Pro Football Hopefuls

Jonathan Lee, director of sports performance technology in Intel’s Olympic Technology Group, has flown 11 hours and quarantined for 14 days to stitch skeletons together on a computer. And if it works as well as he hopes, it’ll be an amazing innovation for action replays at the 2020 Toyko Olympics.

“Part of our artificial intelligence is [designed] to stitch together the right skeletons when you have up to eight or nine athletes running down the track,” Lee told Digital Trends from his hotel room in Tokyo’s Olympic Village, 5,000 miles from his home in San Francisco.

He is part of a crack team of Intel engineers that have been dispatched to the Tokyo Games to incorporate Intel’s 3D Athlete Tracking (or 3DAT, pronounced “three-dat”) technology into this year’s Olympic broadcasts. 3DAT’s breathtaking overlay visualizations will be made available during replays of the 100-meter, 200-meter, 4×100-meter relay, and hurdle athletic events, taking place between July 30 and August 4.

Crack team of Intel engineers setting up 3DAT equipment at Olympic Trials.
Intel

“3D Athlete Tracking … is a technology we have developed here at Intel that allows us to take standard video of athletes and extract information about their form and motion,” said Lee. “We do this using A.I. and computer vision. [Using our technology, we can] recognize the different parts of the body, from the eyes and nose all the way down to the ankles and toes, and use this to construct a 3D skeleton of the athlete or, in some cases, multiple athletes. From those skeletons, we can then extract information like velocity, acceleration, and biomechanics.”

To put it simply, 3DAT fuses together video taken from multiple 4K machine vision cameras with broadcast footage and uses this to create a three-dimensional model of Olympians in action. These can be used to provide computer-generated replays of the action. What gives this the edge over traditional video replays, however, is that the 3DAT technology can ingest its various sources of video footage and use this to generate motion-capture models that can be rotated in 3D space.

“You give the [broadcaster] the ability to, in essence, rotate, zoom, put the camera wherever they want to,” said Lee.

Heat maps and beyond

It’s not just rotating the “camera,” either. By extracting data like velocity and acceleration from the 3D models, 3DAT can overlay the models with added information such as heat maps to indicate how fast an athlete is running, when they hit their top speed, and how long they’re able to maintain this speed. It’s a level of eye-catching data visualization never before tried at the Olympic Games — or just about anywhere else, for that matter.

Sha'Carri Richardson makes a STATEMENT with dominant 100m heat at trials | NBC Sports
“What you want is something that is useful and beautiful and helps the viewer at home really connect up with the athletes and understand something that they didn’t know before,” said Lee.

Motion capture is, of course, nothing new. It’s been used in Hollywood for years, most notably in some of the astonishing theatrical performances captured by companies like Weta Digital from actors like Andy Serkis (who has played everyone from Gollum in The Lord of the Rings to Caesar in Dawn of the Planet of the Apes to King Kong in, err, King Kong). Mo-cap is also frequently used in the gaming world to ensure that on-screen avatars move as closely as possible to real people. But while many mo-cap suits are studded with sensors to capture the movement of individual limbs, 3DAT requires zero sensors.

The problem, Lee said, is that while motion capture suits are fine for certain scenarios, tracking elite-level athletes is not necessarily among them.

“Imagine you put a sensor on someone’s head, elbow, chest, and then you say, ‘All right, go do a high jump, and when you land, you’re gonna feel all these sensors being pressed into your body,’ right?” he said. “You can imagine that’s not necessarily a pleasant experience. Or [how about] a sprinter? If the sensor is placed right below the knee, that’s going to interfere with how they come out of the blocks and how they run.”

Instead, 3DAT relies entirely on computer vision and pose estimation algorithms to analyze the biomechanics of athletes’ movements. Lee said that this can be done accurately enough to capture even the smallest nuances of an athlete’s motion. No trackers required.

The future of 3DAT

Crack team of Intel engineers siting in front of monitors using 3DAT at Olympic Trials.
Intel

With this in mind, Lee doesn’t just see 3DAT as a data-viz tool for entertaining and informing the viewers at home. It’s also being used as a training tool for athletes to review their performance. “We’ve had three different elite-level coaches use the exact same phrase to describe this as the ‘holy grail of coaching,’” he said. One potentially transformative use case? Helping to diagnose injuries.

“One of the biggest things that affect athletes are soft tissue injuries,” Lee said. “These usually start to show up in the form of asymmetry. So perhaps your left and right stride length might become different, or the way that your hips move might [change], right? These are things that can happen even before an injury occurs. If you start to look at an athlete a little more longitudinally — say, at the beginning of the season, during, after, maybe during some sort of functional movement assessment — [an A.I.] can take over to recognize precursors for injuries, [so that the coaches and athletes] can ward them off before they happen.”

For this reason, Lee sees the future of 3DAT being increasingly wrapped up with artificial intelligence. “The question that comes up is what do you do with all this data?” he noted. “This is where I see the next frontier for 3DAT … There has to be a next level, whether that’s for injury prevention or for improving performance or helping rehabilitation … That next question is really the one that we’ll need to answer to be able to turn this from a technology that’s [really] cool into a technology that’s cool and [incredibly] useful.”

For now, though, Olympics viewers will have to settle for just “really cool.” Something tells us that will probably be enough. Coming soon to a gold medal event near you (or, at least, on your television).

Luke Dormehl
Former Digital Trends Contributor
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Rainbow Six Extraction will turn you into a better squad mate
A squad attacks an alien in Rainbow Six Extraction.

My first Rainbow Six Extraction mission didn’t quite go as planned.

During a demo event, I was paired up with two strangers who would become my close squadmates for nearly four hours. Having completed a solo tutorial with little trouble, I went into our inaugural mission thinking I’d carry my team to victory as a party of one. One swarm of slimy aliens later, I was out of action, cocooned in a sickly yellow shell. My poor teammates had to abort the mission, toss my immobilized body on their shoulders, and extract me to safety. I was quite literally dead weight.

Read more
Everything LG announced for the smart home at CES 2022
LG room interior with sofa and drawer with mockup poster near window.

LG has almost always had a presence in the smart home space, but the company made that clearer than ever before at CES 2022 with a flurry of new smart home devices aimed at improving people's lives. From improving our air quality and turning us into the perfect cooks to adding more settings to washers and dryers, the LG ThinQ app plays a larger role than ever before.
PuriCare AeroTower

The company aims to not only improve your air quality, but also your comfort with the LG PuriCare AeroTower. Not your ordinary air purifier, the PuriCare AeroTower is also a fan and a heater. The model supports three airflow modes (with 10 levels of intensity each): Wide, Diffusion, and Ddirect. Depending on the model, you can add heat (up to 86 degrees Fahrenheit) to any airflow mode. The unit uses multistage True HEPA filters to effectively capture most particles floating around a room. It also uses ultraviolet-C light to help combat bacteria. For those who want to check on the air quality, they can look on the device or check out the LG ThinQ app on their phone.
InstaView Double Oven Range

Read more
Why Matter is the most important smart home trend from CES 2022
A collection of Matter-enabed Amazon devices.

If you've been following our CES 2022 coverage, then you've seen one particular name crop up around almost every smart home announcement: Matter. The Matter movement is built around the concept of a single, unifying standard for smart home devices. It seeks to simplify the operation and setup of the smart home, tearing down the proverbial walled gardens that have risen up in the market.

At the present moment, more than 220 different companies -- including the big three in smart home of Amazon, Google, and Apple -- have all joined the Matter Alliance and pledged their commitment to an open-source approach to development. The number of companies was only 180 in May. In the space of seven months, more than 40 other companies have joined the push. In total, there are more than 2,400 engineers across all these companies working to make the smart home that much smarter. That's a lot of support.

Read more