Skip to main content

Intel’s incredible athlete-tracking A.I. is the ‘holy grail’ of training tech

Intel and EXOS Pilot 3D Athlete Tracking with Pro Football Hopefuls

Jonathan Lee, director of sports performance technology in Intel’s Olympic Technology Group, has flown 11 hours and quarantined for 14 days to stitch skeletons together on a computer. And if it works as well as he hopes, it’ll be an amazing innovation for action replays at the 2020 Toyko Olympics.

“Part of our artificial intelligence is [designed] to stitch together the right skeletons when you have up to eight or nine athletes running down the track,” Lee told Digital Trends from his hotel room in Tokyo’s Olympic Village, 5,000 miles from his home in San Francisco.

He is part of a crack team of Intel engineers that have been dispatched to the Tokyo Games to incorporate Intel’s 3D Athlete Tracking (or 3DAT, pronounced “three-dat”) technology into this year’s Olympic broadcasts. 3DAT’s breathtaking overlay visualizations will be made available during replays of the 100-meter, 200-meter, 4×100-meter relay, and hurdle athletic events, taking place between July 30 and August 4.

Crack team of Intel engineers setting up 3DAT equipment at Olympic Trials.
Intel

“3D Athlete Tracking … is a technology we have developed here at Intel that allows us to take standard video of athletes and extract information about their form and motion,” said Lee. “We do this using A.I. and computer vision. [Using our technology, we can] recognize the different parts of the body, from the eyes and nose all the way down to the ankles and toes, and use this to construct a 3D skeleton of the athlete or, in some cases, multiple athletes. From those skeletons, we can then extract information like velocity, acceleration, and biomechanics.”

To put it simply, 3DAT fuses together video taken from multiple 4K machine vision cameras with broadcast footage and uses this to create a three-dimensional model of Olympians in action. These can be used to provide computer-generated replays of the action. What gives this the edge over traditional video replays, however, is that the 3DAT technology can ingest its various sources of video footage and use this to generate motion-capture models that can be rotated in 3D space.

“You give the [broadcaster] the ability to, in essence, rotate, zoom, put the camera wherever they want to,” said Lee.

Heat maps and beyond

It’s not just rotating the “camera,” either. By extracting data like velocity and acceleration from the 3D models, 3DAT can overlay the models with added information such as heat maps to indicate how fast an athlete is running, when they hit their top speed, and how long they’re able to maintain this speed. It’s a level of eye-catching data visualization never before tried at the Olympic Games — or just about anywhere else, for that matter.

Sha'Carri Richardson makes a STATEMENT with dominant 100m heat at trials | NBC Sports
“What you want is something that is useful and beautiful and helps the viewer at home really connect up with the athletes and understand something that they didn’t know before,” said Lee.

Motion capture is, of course, nothing new. It’s been used in Hollywood for years, most notably in some of the astonishing theatrical performances captured by companies like Weta Digital from actors like Andy Serkis (who has played everyone from Gollum in The Lord of the Rings to Caesar in Dawn of the Planet of the Apes to King Kong in, err, King Kong). Mo-cap is also frequently used in the gaming world to ensure that on-screen avatars move as closely as possible to real people. But while many mo-cap suits are studded with sensors to capture the movement of individual limbs, 3DAT requires zero sensors.

The problem, Lee said, is that while motion capture suits are fine for certain scenarios, tracking elite-level athletes is not necessarily among them.

“Imagine you put a sensor on someone’s head, elbow, chest, and then you say, ‘All right, go do a high jump, and when you land, you’re gonna feel all these sensors being pressed into your body,’ right?” he said. “You can imagine that’s not necessarily a pleasant experience. Or [how about] a sprinter? If the sensor is placed right below the knee, that’s going to interfere with how they come out of the blocks and how they run.”

Instead, 3DAT relies entirely on computer vision and pose estimation algorithms to analyze the biomechanics of athletes’ movements. Lee said that this can be done accurately enough to capture even the smallest nuances of an athlete’s motion. No trackers required.

The future of 3DAT

Crack team of Intel engineers siting in front of monitors using 3DAT at Olympic Trials.
Intel

With this in mind, Lee doesn’t just see 3DAT as a data-viz tool for entertaining and informing the viewers at home. It’s also being used as a training tool for athletes to review their performance. “We’ve had three different elite-level coaches use the exact same phrase to describe this as the ‘holy grail of coaching,’” he said. One potentially transformative use case? Helping to diagnose injuries.

“One of the biggest things that affect athletes are soft tissue injuries,” Lee said. “These usually start to show up in the form of asymmetry. So perhaps your left and right stride length might become different, or the way that your hips move might [change], right? These are things that can happen even before an injury occurs. If you start to look at an athlete a little more longitudinally — say, at the beginning of the season, during, after, maybe during some sort of functional movement assessment — [an A.I.] can take over to recognize precursors for injuries, [so that the coaches and athletes] can ward them off before they happen.”

For this reason, Lee sees the future of 3DAT being increasingly wrapped up with artificial intelligence. “The question that comes up is what do you do with all this data?” he noted. “This is where I see the next frontier for 3DAT … There has to be a next level, whether that’s for injury prevention or for improving performance or helping rehabilitation … That next question is really the one that we’ll need to answer to be able to turn this from a technology that’s [really] cool into a technology that’s cool and [incredibly] useful.”

For now, though, Olympics viewers will have to settle for just “really cool.” Something tells us that will probably be enough. Coming soon to a gold medal event near you (or, at least, on your television).

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Apps to help you start good habits and level up your life in 2022
xiaomi mi 10 pro review apps

Technology played a huge role in our lives in 2021. From healthcare to education, everything happened online through those tiny little icons on our screens. Now as 2022 inches closer, it’s time to rethink our relationship with those glowing, sometimes productive, sometimes distracting boxes. 
As you make your New Year's resolutions, think about how you can use technology to enhance rather than empty your life. For some, this might look like deleting unused, draining apps and for others, it might mean joining productive ones that will improve the quality of your life. 
Need some ideas? Here are the most common apps people are joining in 2022. 
Reading and audiobook apps: Kindle, Kobi, Audible, etc. 
As we’re operating in this information economy, the demand for accessible learning has shot up in recent years. Not everyone has access to in-person classes and libraries (especially during the pandemic), so e-books have played a huge role in filling those gaps. That’s why e-reading apps like Kindle, Kobi, and ePub Reader are in-demand. After all, if you’re going to spend a lot of time on your phone, might as well spend it doing something productive!
“I want to double my reading goal this year,” says Jessica Kats, e-commerce and retail expert at Soxy who spent the lockdown reading 20 books. 

Free reading apps have also helped reduce economic barriers to knowledge. “I have been a bibliophile since a young age, though a few financial constraints held me back from buying paperbacks for a long time,” says Andre Flynn, founder of gadnets.com. Now free reading apps are helping users like Andre access new information and fuel their passion for writing without spending a lot of money on physical books. 
Along with e-books, users are also dipping their toes in the world of audiobooks. People who don’t get a lot of time to sit down and read are consuming information through audiobooks, many of which are now freely available on apps like Audible (free trial), Audiobooks.com, LibriVox, and more. 
Language learning apps: Duolingo, Busuu, Memrise, etc. 
Learning a new language has dozens of benefits, so this new year, users are setting aside time to invest in language learning by installing apps like Duolingo, Busuu, Memrise, and others. 
“Instead of spending half an hour every night scrolling mindlessly through social media, I'm choosing to spend that half-hour learning a new(ish) language and strengthening my brain,” says Brian Donovan, CEO of TimeShatter.

Read more
This is the year 2022, according to old sci-fi movies
2022 predictions sci fi movies retro future nasa

One of the many repercussions of the global pandemic is that it has made planning for the future extremely tough. That tropical vacation you’re saving for this summer? Who knows whether you’ll be able to fly to that country at that time. That mid-January drink with buddies? Better hope nothing changes before then.

However, in the Before Times, science fiction was busy imagining what the world would look like, circa 2022. How accurate were their best efforts? While we've still got 12 full months for all the prophecies to come true, as the New Year commences we can start to assess the accuracy of five movies set in the once-far future world of 2022:
Soylent Green
Soylent Green (1973) Official Trailer - Charlton Heston, Edward G Robinson Movie HD

Read more
Smart home trends to look for in 2022
Astro Robot lifestyle image.

It's a brand-new year, and that means new and exciting innovations in the world of smart home technology. Although no one can say for sure what this year might hold for the ever-expanding industry, we can make a few educated guesses based on the direction it is currently headed, as well as with what we know from CES 2022 so far.
Autonomy and robots
Whether we're ready for it or not, it's clear that the biggest names in the smart home world are devoting a not-insignificant amount of time and research to household robots. While the idea of a semi-intelligent robot that can handle the cleaning around the house isn't unwelcome, it can also be a bit unsettling.

With that in mind, Amazon's Astro will likely see more widespread adoption, as will the Ring Always Home Cam. In 2020, Sunflower Labs launched The Bee, an autonomous home security drone for your yard. Samsung's JetBot AI+ might be a robot vacuum, but its built-in home security features make it closer to a home robot than many other devices on the market.

Read more