This algorithm turns World Cup games into 3D augmented reality

The men’s World Cup kicks off this week. As exciting as it promises to be, however, most of us will only get to watch it on flat television sets, as opposed to getting the three-dimensional experience of seeing it live and in person. Researchers from the University of Washington have come up with an augmented reality alternative, though. While it probably won’t be available to the masses in time for this year’s tournament, it does hint at one way in which fans at home may be able to enjoy sports games in the future.

What the researchers have developed is a machine learning algorithm capable of transforming 2D soccer clips into 3D reconstructions, which may be viewed using AR headsets like Microsoft’s HoloLens. The results allow viewers to turn flat surfaces like their desk or kitchen table into a virtual pitch, complete with three-dimensional action that you can circle around to view from different angles.

It’s not quite the equivalent of watching it in person, but it’s much closer than regular TV. And significantly cheaper, too!

“Our goal is to enhance the viewing experience of sports,” Konstantinos Rematas, one of the researchers on the project, told Digital Trends. “Instead of watching a soccer game or highlights on a flat 2D screen, we convert the original video into 3D and visualize it in augmented reality. Essentially the game becomes a hologram, where you can move around and look from different viewpoints, generating a more immersive experience.”

soccer augmented reality algorithm hologramsoccer
Konstantinos Rematas

As its input, the algorithm generating the AR experience requires just a single YouTube soccer video. Because this is not enough information to train an entire rendering system to “upconvert” 2D players into 3D, the neural network learned to estimate depth by playing the playing the EA video game FIFA 2018. Using the totality of this information, it can do a passable job of accurately gauging where players are on the pitch.

At present, the system is still a work in progress. For instance, the ball is not yet properly rendered (something which turns out to be pretty darn important in soccer) and the players remain two-dimensional cutouts. These are two improvements the researchers hope to make.

“The next steps are about increasing the quality of the game reconstruction,” Rematas continued. “In particular, we want to estimate precisely the location of the ball and reconstruct better the players — occlusions, full 3D shape estimation, [and more].” In addition, the researchers plan to extend the framework to also cover other sports such as basketball, hockey, and football.

A paper describing the work is available to read online.

Movies & TV

After a slow burn, 'Daredevil' season 3 erupts into an inferno

Season 3 of Marvel's Daredevil series on Netflix raises the already-high bar for the show with some of its best drama and action to date within the first six episodes, and some of its most fascinating characters - both new and familiar.
Movies & TV

The best movies on Netflix in October, from 'The Witch’ to ‘Black Panther’

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, subdued humor, or anything in between.
Photography

Apple acquisition points to possibility of easy background removal in photos

According to a Danish newspaper, Apple now owns a company that uses machine learning to remove or replace the background on photos and videos. Spektral's website says it is working to bring the A.I. background removal to smartphones.
Mobile

iOS 12.0.1 fixes iPhone XS charging issue, Wi-Fi bugs, and more

After months of betas, the final version of iOS 12 is here to download. The latest OS comes along with tons of new capabilities, from grouped notifications to Siri Shortcuts. Here are all the features you'll find in iOS 12.
Emerging Tech

New ‘parkour’ video shows Boston Dynamics robot training to overthrow humanity

Robots doing backflips? That's so 2017! In its latest jaw-dropping YouTube video, Boston Dynamics’ Atlas robot pulls off some frankly astonishing parkour stunts for our viewing pleasure.
Emerging Tech

With VR dinosaurs and ‘Minecraft,’ one hospital is making medicine less scary

From augmented reality rabbits on the wards to a Minecraft recreation of the hospital for kids to explore, one of the world's most renowned children's hospitals just got a major tech overhaul.
Emerging Tech

Will we ever fly supersonic again? Unraveling the concorde’s complex legacy

In a new book, Last Days of the Concorde, journalist and author Samme Chittum delves into the mindset that inspired engineers to design this marvel, the series of events that led to its fatal crash, and the possibility that commercial SSTs…
Emerging Tech

Check out the British Army’s beefy new bomb-disposal robot

The British Army is about to get an impressive new explosive ordnance disposal robot that is able to climb stairs, negotiate slopes, cut wires, and … oh, yes, dispose of bombs, too.
Emerging Tech

Kill it before it lays eggs! Crazy 32-leg robot moves like a cyborg sea urchin

We’ve seen one-legged, two-legged, four-legged and even six-legged robots, but researchers from Japan have gone way, way further with their latest project: A 32-legged robot. Check it out.
Emerging Tech

Leafy greens are grown by machines at new, automated Silicon Valley farm

Farming hasn't changed too much for hundreds of years. Now a new startup called Iron Ox has opened its first automated hydroponics farm, producing a variety of leafy greens tended by machines.
Emerging Tech

Awesome Tech You Can’t Buy Yet: DIY smartphones and zip-on bike tires

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Gaming

As deaf gamers speak up, game studios are finally listening to those who can’t

Using social media, personal blogs and Twitch, a small group of deaf and hard-of-hearing players have been working to make their voices heard and improve accessibility in the gaming industry.
Emerging Tech

From flying for fun to pro filmmaking, these are the best drones you can buy

In just the past few years, drones have transformed from a geeky hobbyist affair to a full-on cultural phenomenon. Here's a no-nonsense rundown of the best drones you can buy right now, no matter what kind of flying you plan to do.
Emerging Tech

Get your head in the clouds with the best vaporizers for flower and concentrates

Why combust dead plant matter when you could vaporize the good stuff and leave the leaves behind? Here's a rundown of the best vaporizers money can buy, no matter what your style is.