Skip to main content

This algorithm turns World Cup games into 3D augmented reality

 

The men’s World Cup kicks off this week. As exciting as it promises to be, however, most of us will only get to watch it on flat television sets, as opposed to getting the three-dimensional experience of seeing it live and in person. Researchers from the University of Washington have come up with an augmented reality alternative, though. While it probably won’t be available to the masses in time for this year’s tournament, it does hint at one way in which fans at home may be able to enjoy sports games in the future.

Recommended Videos

What the researchers have developed is a machine learning algorithm capable of transforming 2D soccer clips into 3D reconstructions, which may be viewed using AR headsets like Microsoft’s HoloLens. The results allow viewers to turn flat surfaces like their desk or kitchen table into a virtual pitch, complete with three-dimensional action that you can circle around to view from different angles.

Please enable Javascript to view this content

It’s not quite the equivalent of watching it in person, but it’s much closer than regular TV. And significantly cheaper, too!

“Our goal is to enhance the viewing experience of sports,” Konstantinos Rematas, one of the researchers on the project, told Digital Trends. “Instead of watching a soccer game or highlights on a flat 2D screen, we convert the original video into 3D and visualize it in augmented reality. Essentially the game becomes a hologram, where you can move around and look from different viewpoints, generating a more immersive experience.”

Konstantinos Rematas

As its input, the algorithm generating the AR experience requires just a single YouTube soccer video. Because this is not enough information to train an entire rendering system to “upconvert” 2D players into 3D, the neural network learned to estimate depth by playing the playing the EA video game FIFA 2018. Using the totality of this information, it can do a passable job of accurately gauging where players are on the pitch.

At present, the system is still a work in progress. For instance, the ball is not yet properly rendered (something which turns out to be pretty darn important in soccer) and the players remain two-dimensional cutouts. These are two improvements the researchers hope to make.

“The next steps are about increasing the quality of the game reconstruction,” Rematas continued. “In particular, we want to estimate precisely the location of the ball and reconstruct better the players — occlusions, full 3D shape estimation, [and more].” In addition, the researchers plan to extend the framework to also cover other sports such as basketball, hockey, and football.

A paper describing the work is available to read online.

Luke Dormehl
Former Digital Trends Contributor
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
YouTuber claims that this ugly 3D-printed mouse is actually the best for gaming
OptimumTech's Zeromouse alongside a regular mouse.

According to the YouTuber who made it, this 3D-printed mouse is one of the best gaming mice in the world -- but boy, does it look odd. OptimumTech designed the Zeromouse from the ground up by modding a Razer mouse with a 3D-printed shell that made it a lot more lightweight, and reportedly, more ergonomic.

The end result weighs just 25 grams, all thanks to the custom-made lightweight shell. The internals of the mouse belong to the Razer Viper V2 Pro, and OptimumTech doesn't seem to have made any changes to that, but the outside looks drastically different. In all honesty, it kind of looks like what would happen to a regular gaming mouse if you dropped it from three stories up and it fell apart.

Read more
Meet the Qualcomm chips powering Meta’s Quest 3 headset
Alan Truly holds a display model of Qualcomm's Snapdragon XR2 Gen 2 chip.

A display model of Qualcomm's Snapdragon XR2 Gen 2 chip. Tracey Truly / Digital Trends

Qualcomm just announced two new Snapdragon chips, one that dramatically increases the performance available to next-generation VR headsets, and another that expands the capabilities of new smart glasses.

Read more
Apple’s Vision Pro could get this incredible gaming upgrade
A person plays a game on the Apple Vision Pro.

Apple has touted its Vision Pro headset as a premier gaming device that works with many different controllers. But with extensive gesture control seemingly used everywhere on the product except for gaming, it feels like a missed opportunity.

Well, that could change in the future, as a newly published Apple patent (number 11709554) shows how future editions of the Vision Pro could transform your hands into gaming controllers, complete with fingertip controls and haptic feedback.

Read more