Skip to main content

Watch Nvidia’s powerful A.I. change day into night, and winter into summer

nvidia ai winter summer car nips research street scene
Image used with permission by copyright holder
Artificial intelligence is so awesome these days that it can turn summer into winter, and day into night. Well, in a video at least. Presented at this week’s Conference on Neural Information Processing Systems (NIPS), researchers from Nvidia showcased some seriously smart machine-learning tools that are able to digitally alter a video of a winter scene so that it looks like it was shot on a sunny July 4 weekend.

We’re not referring to a simple palette swap, either; we’re talking about eliminating roadsides lined with snow and replacing them with grassy banks. The only thing that’s missing is a family out having a barbecue.

Snow2SummerImageTranslation-04

“The goal of our research is to give machines the ability to create or ‘imagine’ scenes on their own,” Ming-Yu Liu, a senior research scientist at Nvidia, told Digital Trends. “This is a difficult challenge, because most A.I. today require you to have images as training data that exactly correspond for both the input and target image. Let’s say you wanted an A.I. that could turn a driving video from night into day, or convert a sunny day into a rainy day. Today, you would need to record video of that street during both daytime and nighttime, shot from exactly the same location, with the objects — vehicles, trees, pedestrians — in exactly the same location. In contrast, our method just needs a set of daytime images and another set of nighttime images for training, and these images can be taken in different cities or countries. Without the requirement of corresponding images, collecting data for training our model is much easier.”

To create their image-altering tool, the Nvidia researchers developed a novel neural network design to achieve unsupervised image-to-image translation. The algorithm and its source code is described in a paper available here. While it’s certainly an impressive tech demo, Liu points out that it has numerous real-world applications. For example, it could be immensely useful in video editing work. However, Nvidia has a much more immediate application in mind: Training self-driving cars.

Day2NightImageTranslation-03

“We did this research to help train self-driving cars under different weather and lighting conditions,” he said. “You can shorten training time for self-driving cars by teaching them with simulation. Using our technique, we can convert daytime to nighttime video, add rain or snow, and use that to help train self-driving cars [to deal with a wide range of scenarios they might face].”

Next up, the team wants to work to improve the robustness of the technology, while finding even better ways of improving data efficiency — allowing them to train their neural network with less data.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
New self-driving car algorithm keeps you safe by constantly predicting doom
waymo self driving car testing

Call it fatalistic, pessimistic, or just really, really, smart, but a new self-driving car algorithm developed by researchers at Germany’s Technical University of Munich (TUM) thrives on thinking about the worst thing that could happen at every moment. And then figuring out how to get out of it without endangering or obstructing traffic.

“Current autonomous driving systems usually incorporate most-likely evolutions of a traffic scenario, [such as] the preceding vehicle will most likely accelerate,” Christian Pek, a researcher in the university's researcher in the Cyber-Physical Systems Group, told Digital Trends. “However, this design might result in unsafe behaviors if traffic participants behave differently than expected -- for example, [if instead] the preceding vehicle decelerates. Our algorithm addresses this problem by computing all possible future evolutions of the scenario by considering all possible motions of other traffic participants that are compliant with traffic rules. As a result, we are able to ensure that decisions are safe regardless of the future legal motion of other traffic participants.”

Read more
Deep-learning A.I. is helping archaeologists translate ancient tablets
DeepScribe project 1

Deep-learning artificial intelligence is helping grapple with plenty of problems in the modern world. But it also has its part to play in helping solve some ancient problems as well -- such as assisting in the translation of 2,500-year-old clay tablet documents from Persia's Achaemenid Empire.

These tablets, which were discovered in modern-day Iran in 1933, have been studied by scholars for decades. However, they’ve found the translation process for the tablets -- which number in the tens of thousands -- to be laborious and prone to errors. A.I. technology can help.

Read more
Mayflower Autonomous Ship is headed to sea to test its self-driving boat tech
mayflower ship seat tests ibm

Entering the Mind of the Mayflower

Last year, Digital Trends wrote about an ambitious project that seeks to dispatch an unmanned autonomous ship across the Atlantic Ocean on the 400th anniversary of the voyage of the Mayflower. While this first-of-its-kind journey won’t take place until the this fall, IBM and marine research organization Promare announced Thursday that the “A.I. Captain” that will power the self-steering vessel is ready to go to sea for a month of testing. This trial, which will take place on a manned research vessel off the coast of Plymouth in the U.K., will test out the onboard A.I. and edge computing system to see how well it navigates.

Read more