Machine learning system brings still images to life by guessing what comes next

machine learning images to life fig5
Whether it’s guessing which songs you want to listen to or which ads you should be shown, modern AI is increasingly focused on predicting the future. But there’s an enormous gulf between those kind of applications and looking at a scene and guessing what will happen next.

That’s what researchers at the Massachusetts Institute of Technology have done, with a new paper revealing not just their ability to look at still images and guess what will happen next — something we’ve covered in the past — but to actually generate video of it.

“What we’re interested in is teaching machines what can happen in a particular setting,” Carl Vondrick, a Ph.D. student in computer science, told Digital Trends. “For example, we wanted a machine to recognize what happens on a beach. We want it to know that waves are going to crash, people are going to play in the water — these are all things it’s very difficult to teach a machine. The reason is that it would be very time-consuming for a person to sit down and write rules to explain everything that can happen in any given scenario. What we wanted to do was to teach them from watching massive amounts of video instead.”

To do this, Vondrick and his fellow researchers used deep learning to “train” a computer to understand everyday scenes by watching the equivalent of two years’ worth of online video footage. After that, they then showed the machine still images and asked it to generate video to bring that scene to life.

Like many examples of computation creativity, such as Google’s DeepDream technology, the generated images can be a bit trippy — although there’s no denying they get a lot right about the world. MIT’s system, for instance, takes a still image of a beach and animates it to show waves crashing, just as Vondrick hoped it would.

Limitations mean that the videos are only one second long, and animated like a made-for-TV movie from the 1990s, but it’s an impressive start.

“One application I’m really excited about is predicting the future,” Vondrick said. “What our work is doing is to generate videos letting us anticipate what is going to happen next. This is important for robotics. You can imagine a robot being able to predict when an elderly person is going to fall down, for example, and then preventing that from happening. It’s also got application in computer graphics. Right now, the system isn’t photorealistic, but as we keep improving it I think we’ll be able to generate full-length videos. This could therefore be very useful as a tool in something like Hollywood movie production.”

It’s potentially even more significant than that. “In the largest sense, this work is about a computer learning to simulate the world,” he said. “Getting a machine to understand how the world works will be very important as computers start taking more and more actions in our world — and need to understand what is going on in order to do that.”


Midrange phones can’t do A.I., but MediaTek’s P90 chip aims to change that

MediaTek has announced the Helio P90 mobile processor, which it says will bring the best A.I. features we see on high-end smartphones, to the mid-range. We spoke to the company about the chip.

Nvidia’s Jetson AGX Xavier module is designed to give robots better brains

Nvidia's pricey Jetson AGX Xavier might help drive the next generation of smart robots. Nvidia hopes that developers will use its new Xavier module to power AI-driven machines like delivery drones and robots used in manufacturing.
Cars is helping Hyundai mine the artificial intelligence gold rush

In November 2018, Hyundai invested in a startup named We talked to the company's founder to learn more about what that means for consumers in the not-too-distant futures.
Movies & TV

The best shows on Netflix, from 'Haunting of Hill House’ to ‘Twilight Zone’

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.
Emerging Tech

With this robotic garage, retrieving your car is like using a vending machine

Remembering where we parked our cars can be a real pain. But what if our cars came to find us, rather than the other way around? A new automated robot parking valet system aims to help.

Best Products of 2018

Our reception desk has so many brown boxes stacked up, it looks like a loading dock. We’re on a first-name basis with the UPS guy. We get new dishwashers more frequently than most people get new shoes. What we’re trying to say is: We…
Emerging Tech

A lidar-equipped truck knows exactly how much de-icer to apply on roads

Lidar is best known as the laser-based technology that helps self-driving cars sense their surroundings. But the city of Knoxville has another, more seasonal use for it: De-icing roads.
Emerging Tech

This unusual nature-inspired robot is equally at home on land or in the water

This intriguing, nature-inspired robot may look unusual, but it's impressively capable of moving on both land and water without problem. Heck, it can even travel on ice if necessary.
Emerging Tech

This cryptocurrency wallet for kids isn’t nearly as stupid as it sounds

So you’ve taught your 6-year-old child to read, write, and play nice with others. What’s next? Give them a base understanding of cryptocurrency, of course. This Kickstarter aims to help.
Emerging Tech

Light, speed: Lighting kit for DJI Mavic 2 lets you fly and film in the dark

Lume Cube, maker of small battery-powered LED lights for mobile photography, has announced a new lighting kit built specifically for the DJI Mavic 2 -- the first of its kind. Already our favorite drone, this makes the Mavic 2 even better.
Emerging Tech

Prepare for liftoff: Here are all the important upcoming SpaceX rocket launches

From ISS resupply missions to a host of communication and scientific satellite launches, SpaceX has a busy year ahead. Here's a rundown of some of the company's most important missions slated for the next year.
Emerging Tech

Virgin Galactic’s latest test flight takes it to the edge of space

Sir Richard Branson's Virgin Galactic has successfully carried out its fourth powered flight in Mojave, California. It was the company's most ambitious test flight yet -- and bodes well for the future.
Emerging Tech

We’re going to the Red Planet! All the past, present, and future missions to Mars

SpaceX isn't the only organization pining to visit the Red Planet. Here's a detailed list of all operational and planned missions to Mars, along with explanations of their objectives, spacecraft details, and mission proposals.
Emerging Tech

There’s a giant EMP blaster in New Mexico. Don’t worry, it’s here to protect us

An electromagnetic pulse has the potential to disable virtually all electronics within a large area. To help protect against such a threat is a new, friendly EMP emitter. Here's how it works.