Machine learning system brings still images to life by guessing what comes next

machine learning images to life fig5
Whether it’s guessing which songs you want to listen to or which ads you should be shown, modern AI is increasingly focused on predicting the future. But there’s an enormous gulf between those kind of applications and looking at a scene and guessing what will happen next.

That’s what researchers at the Massachusetts Institute of Technology have done, with a new paper revealing not just their ability to look at still images and guess what will happen next — something we’ve covered in the past — but to actually generate video of it.

“What we’re interested in is teaching machines what can happen in a particular setting,” Carl Vondrick, a Ph.D. student in computer science, told Digital Trends. “For example, we wanted a machine to recognize what happens on a beach. We want it to know that waves are going to crash, people are going to play in the water — these are all things it’s very difficult to teach a machine. The reason is that it would be very time-consuming for a person to sit down and write rules to explain everything that can happen in any given scenario. What we wanted to do was to teach them from watching massive amounts of video instead.”

To do this, Vondrick and his fellow researchers used deep learning to “train” a computer to understand everyday scenes by watching the equivalent of two years’ worth of online video footage. After that, they then showed the machine still images and asked it to generate video to bring that scene to life.

Like many examples of computation creativity, such as Google’s DeepDream technology, the generated images can be a bit trippy — although there’s no denying they get a lot right about the world. MIT’s system, for instance, takes a still image of a beach and animates it to show waves crashing, just as Vondrick hoped it would.

Limitations mean that the videos are only one second long, and animated like a made-for-TV movie from the 1990s, but it’s an impressive start.

“One application I’m really excited about is predicting the future,” Vondrick said. “What our work is doing is to generate videos letting us anticipate what is going to happen next. This is important for robotics. You can imagine a robot being able to predict when an elderly person is going to fall down, for example, and then preventing that from happening. It’s also got application in computer graphics. Right now, the system isn’t photorealistic, but as we keep improving it I think we’ll be able to generate full-length videos. This could therefore be very useful as a tool in something like Hollywood movie production.”

It’s potentially even more significant than that. “In the largest sense, this work is about a computer learning to simulate the world,” he said. “Getting a machine to understand how the world works will be very important as computers start taking more and more actions in our world — and need to understand what is going on in order to do that.”

Movies & TV

How Avengers: Infinity War’s Oscar-nominated VFX team made Thanos a movie star

The purple-skinned Thanos proved to be a breakout character in Avengers: Infinity War, thanks to the work of actor Josh Brolin and visual effects studios Digital Domain and Weta. Here's how they brought him to life and earned the film an…
Movies & TV

The best shows on Netflix right now (February 2019)

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.
Cars

Researchers teach self-driving cars to predict pedestrians’ next moves

University of Michigan researchers are developing a system that teaches self-driving cars to predict pedestrian movement. Humans don't always act in their own best self-interest, so autonomous cars will need to practice protective driving.
Emerging Tech

Awesome Tech You Can’t Buy Yet: Grow veggies indoors and shower more efficiently

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Health & Fitness

Immune cell discovery takes us one step closer to a universal flu vaccine

A group of international researchers have made a discovery which could take us one step closer to the universal, one-shot flu vaccine that people around the world have been dreaming of.
Photography

NASA celebrates Earth’s incredible natural beauty with free photo book

NASA has published a fabulous new book featuring stunning imagery captured by its satellites over the years. A hardback version is available for $53, though it can also be downloaded to ebook readers for free, and enjoyed online.
Deals

This new all-in-one flashlight is a power bank, lighter, and screwdriver

The Pyyros modular flashlight can perform numerous field tasks, from hammering to starting fires. If you back it on Kickstarter now, you can score some savings on this innovative flashlight and multi-tool, but act fast: This early-bird…
Movies & TV

Hilarious new Kickstarter aims to fix Scorcese’s last scene in The Departed

A fan of The Departed and apparent hater of rat-as-symbolism imagery has launched a Kickstarter campaign to digitally erase the rodent from the end of Martin Scorsese’s 2006 movie.
Emerging Tech

Baristas beware, Bbox cafe uses robots to brew your morning coffee

Want your morning coffee and pastry prepared by robot? Bbox, a new coffee shop in downtown Berkeley, California, lets customers place their order by app and then uses automation to take care of the rest.
Emerging Tech

This ridiculous new flamethrower makes Elon Musk’s look like a cigarette lighter

The XL18 Flamethrower is a flame-shooting beast on steroids, capable of firing off bursts of flame more than 110 feet in length. The best part? You can order it over the internet today.
Emerging Tech

SpaceX just nailed its most challenging Falcon 9 rocket landing to date

If you've been following the SpaceX launch calendar, you know this week marks the first launch from Cape Canaveral in two months. We have the details on where you can watch the launch live.
Emerging Tech

Touchdown! Japan successfully lands its Hayabusa2 spacecraft on asteroid Ryugu

Japan's space agency has just completed the latest stage of its extraordinarily complex mission, successfully landing its Hayabusa2 spacecraft on an asteroid millions of miles from Earth.
Emerging Tech

Delivery drones: NASA to test advanced traffic control system for cities

Delivery drone services are edging closer as NASA prepares to demonstrate its advanced drone traffic management system, which it claims offers safe and effective control of autonomous aircraft in urban areas.
Emerging Tech

Kickstarter campaign aims to help make 3D-printed space habitats for Mars

Mars X-House is an ambitious project that's intended to create a prototype future Mars habitat using 3D printing. And, thanks to a new Kickstarter campaign, you can be a part of it.