MIT algorithm can predict the (immediate) future from still images

Humans still can’t predict elections but we’re pretty good at predicting the immediate future. Baby drops glass cup, cup falls and shatters, and baby starts to cry. We’re so good at these short-term forecasts that we can often even describe what events will happen next in an image.

But what’s second nature for us can prove complicated for computers. Will the glass break or bounce? Will the baby laugh or cry?

A team of researchers from the Massachusetts Institute of Technology (MIT) Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a system that can predict the following events in images and generate videos to depict them. The system needs work — its current productions are simple, short, and unassuming — but it stands out for its unique approach and accuracy.

“Instead of building up scenes frame by frame, we focus on processing the entire scene at once,” Carl Vondrick, PhD at MIT CSAIL and lead author of the paper, told Digital Trends.

Alternative computer vision models that attempt the same task use recurrent networks to generate predictive videos on a frame-by-frame basis. The system developed by Vondrick and his team uses “convolutional networks” to generate all 32 frames simultaneously.

“The existing approach of going frame by frame has a certain logic,” Vondrick said, “but it also creates a massive margin for error. It’s sort of like a big game of ‘Telephone,” which means that the message most likely will fall apart by the time you go around the whole room.

“In contrast, our approach is the ‘Telephone’ equivalent of speaking to everyone in the room at once,” he added.

The researchers trained the system on a year of footage packed into two million videos and — in order to generate all frames at once — taught it distinguish foregrounds from backgrounds, and mobile objects from stationary ones. They then showed the system still images and had it generate short clips of subsequent events.

Once the system could generate video clips, Vondrick and his team set out to refine it through a method called adversarial learning.

“The idea behind adversarial learning is to have two neural networks compete against each other,” Vondrick said. “One network tries to decide what is real versus fake, and another tries to generate something that fools the first network.”

Through this computer competition the generative algorithm improved the accuracy of its video clips until it was able to fool human subjects 20 percent more often than a baseline model, according to a paper that will be presented next week at the Neural Information Processing Systems conference in Barcelona.

But with accuracy comes complexity and with complexity comes obstacles.

The current system’s videos are short — a mere one and a half seconds long. If the clips were much longer than that, they’d risk their consistency. “The key challenge is being able to reliably track the relationships between all of the objects in a scene … to make sure that the video that’s being generated still makes sense five or ten seconds later,” Vondrick said. To develop accurate and long videos, the system may need human input to help it grasp context and connection between seemingly unrelated actions, such as jogging and showering.

Vondrick’s ambitious end goal is to develop an algorithm that can create believable feature-length films, though he admits that is still some years off. In the near term though he thinks this system could refine AI systems by helping them adapt to unpredictable environments.

Home Theater

Sony’s 360 Reality Audio is the epic sound revolution you didn’t know you needed

After Sony’s utterly bizarre press conference, I almost missed what was perhaps the most impactful sonic experience at the show. Luckily, I went back to Sony’s booth on the last day of the show, only to have my mind blown.

Do these case images confirm a side-mounted fingerprint scanner on the S10 E?

It won't be long now; With 2019 underway, the Samsung Galaxy S10 is almost here. Before it arrives, here's absolutely everything you need to know about all three of Samsung's next flagships.

The hottest Nintendo Switch games you can get right now

The Nintendo Switch's lineup started off small, but games have steadily released as the console continues through its second year. Here are the best Nintendo Switch games available now.

This list of PlayStation 4 exclusives puts its competitors to shame

The PlayStation 4's game library and incredible selection of exclusive games could make anyone with an Xbox One or Nintendo Switch think twice. Here's our list of the latest and greatest PS4 exclusives.

It's not all free money. Here's what to know before you try to mine Bitcoin

Mining Bitcoin today is harder than it used to be, but if you have enough time, money, and cheap electricity, you can still turn a profit. Here's how to get started mining Bitcoin at home and in the cloud.

Need a free alternative to Adobe Illustrator? Here are our favorites

Photoshop and other commercial tools can be expensive, but drawing software doesn't need to be. This list of the best free drawing software is just as powerful as some of the more expensive offerings.
Emerging Tech

Awesome Tech You Can’t Buy Yet: camera with A.I. director, robot arm assistant

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!

What is fixed wireless 5G? Here’s everything you need to know

Here's fixed wireless 5G explained! Learn what you need to know about this effective new wireless technology, when it's available, how much it costs, and more. If you're thinking about 5G, this guide can help!

Fix those internet dead zones by turning an old router into a Wi-Fi repeater

Is there a Wi-Fi dead zone in your home or office? A Wi-Fi repeater can help. Don't buy a new one, though. Here is how to extend Wi-Fi range with another router you have lying around.

Heal your wrist aches and pains with one of these top ergonomic mice

If you have a growing ache in your wrist, it might be worth considering ergonomic mice alternatives. But which is the best ergonomic mouse for you? One of these could be the ticket to the right purchase for you.

These are the best indie games you can get on PC right now

Though many indie games now come to consoles as well, there's still a much larger selection on PC. With that in mind, we've created a list of the best indie games for PC, with an emphasis on games that are only available on PC.

Want a MacBook that will last all day on a single charge? Check these models out

Battery life is one of the most important factors in buying any laptop, especially MacBooks. Their battery life is typically average, but there are some standouts. Knowing which MacBook has the best battery life can be rather useful.

Want a Dell laptop with an RTX 2060? Cross the new XPS 15 off your list

The next iteration of Dell's XPS 15 laptop won't come with an option for an RTX 2060, according to Alienware's Frank Azor. You could always opt for a new Alienware m15 or m17 instead.

Always have way too many tabs open? Google Chrome might finally help

Google is one step closer to bringing tab groups to its Chrome browser. The feature is now available in Google's Chrome Canady build with an early implementation that can be enabled through its flag system.