Skip to main content

How Nvidia is helping autonomous cars simulate their way to safety


Imagine that you’re the driver of a four-door family sedan approaching a stop sign. When you reach the stop sign, you notice a bicyclist trying to cross the road. Through eye contact, facial expression and body language cues, the bicyclist negotiates their right of way with you. As a result, you decide to let the bicyclist cross the road first, before you proceed to cautiously enter the intersection.

In the autonomous driving world today, there would be no way to “tag” or categorize such an event, said Cognata CEO Danny Atsmon. Current methods allow you to visually identify the bicyclist, but training systems to recognize and understand complex negotiations on the road remain a challenge for the $10.3 trillion autonomous driving industry.

In fact, autonomous driving represents “the single hardest computing problem the world has ever encountered,” as NVIDIA CEO Jensen Huang admitted when he unveiled some of the world’s most powerful graphics processors during the GTC 2018 keynote in San Jose, California.

Bridging the Real and the Virtual

“The world drives 10 trillion miles per year,” Huang said in a pointed presentation — but Atsmon pointed out that self-driving cars only covered three million miles of roads last year. For self-driving vehicles to drive better, they must learn more, and that’s fundamentally the largest challenge faced by the industry. To train an autonomous driving system to have the competence of a human driver, computers would need to drive roughly 11 billion miles, Atsmon told us.

It’s the single hardest computing problem the world has ever encountered.

That figure is calculated based on the 1.09 fatalities per 100 million miles driven in 2015. “So, to say a machine could have as safe a performance as a human being with 95 percent of confidence, you would need to validate for 11 billion miles,” said Atsmon.

Aside from the time needed to reach that goal, there’s also the expense to consider. Right now, the cost per mile for operating an autonomous car is in the hundreds of dollars — accounting for engineering time, data collection and tagging, insurance costs, and the time of a driver to sit in the cockpit of a car. Multiply that by the 11-billion-mile benchmark, and the massive expensive associated with training autonomous cars becomes clear.

Validation is key, and recent accidents involving autonomous vehicles show that incomplete data tests and training scenarios can prove fatal. In one less extreme example, a self-driving shuttle in Las Vegas was navigating at around 0.6 miles per hour, but it crashed into a truck (Jeff Zurschmeide, a freelancer contributor to Digital Trends, was there when it happened). No one was injured, but the puzzling scenario happened because the truck was pulling forward, then backing up as it tried to park. The cause for the crash, according to Atsmon, is that the shuttle wasn’t validated for this type of situation, and it didn’t know what to do — so it proceeded forward slowly and crashed.

Better Simulation for Deeper Learning

The industry’s current solution to bridging the 11-billion-mile gap for autonomous systems to reach human driving competency is to develop simulations to allow cars to learn faster by combining deep learning with a virtual environment.

“Simulation is the path to billions of miles,” Huang said at GTC. Late last year, Alphabet-owned Waymo unveiled Carcraft, its approach to learning by simulation.

Cognata is using the latest advancements in graphics and sensor hardware to create more life-like and realistic models of the world for autonomous cars to learn from. For the computing brains of a self-driving car, it’s like entering a video game modeled on the real world, and that could lead to more realistic driving scenarios to test and validate car driving data. The company has recently mapped out select cities, like San Francisco, using data from GIS — high definition cameras and sophisticated computer algorithms that runs over satellite and street view imagery, resulting in a photo-realistic scene.

Simulation is the path to billions of miles.

To further improve simulations, Nvidia, and some of its partners, are using data from the sensors of autonomous vehicles to build higher definition maps. When autonomous vehicles hit the road, these machines will not only rely on the data that is available through training, but also contribute to data collection by sharing the data that it has captured from its LIDAR, IR, radar, and camera arrays.

When this newly captured data is combined through deep learning with existing low-quality data sets, it will make streets and roads look more photo-realistic. Cognata claims that its algorithms can process the data in a way to bring out details in shadows and highlights, much like an HDR photo from your smartphone’s camera, to create a high-quality scene.

Cognata - Deep Learning Autonomous Driving Simulator

While simulation is an excellent tool, Atsmon noted it has its own flaws. It’s too simple, and for autonomous driving to be realistic, it must learn from edge cases. Cognata claims that it only takes a few clicks to program in an edge case to validate autonomous vehicles for more unusual driving scenarios. Companies building autonomous vehicles will have to be diligent in their search for edge cases that can trick self-driving cars, and creative in crafting solutions for them.

When Self-Driving Fails

Safety is so paramount to autonomous vehicles that Nvidia considers it the single most important thing for the industry. When things fail, fatalities can and do occur, as was recently proven when an autonomous Uber struck and killed a pedestrian in Arizona.

“I can assure you that [Uber is] equally crushed at what happened.”

When questioned in a press meeting about the Uber crash — Uber is a partner of Nvidia — Huang deferred to the ride-sharing company for comments, saying that “we should give Uber a chance to understand what has happened and to explain what has happened.”

“I can assure you that [Uber is] equally crushed at what happened,” Huang added.

Because Nvidia develops an end-to-end solution for autonomous driving, different partners — from Uber to Toyota and Mercedes Benz — may utilize all or some parts of the system. “There are some 370 companies around the world who are using our technologies in some way.” At the show, Nvidia also announced Orin, the next-generation computer of its DRIVE platform.


Humans as a Backup

While self-driving cars are getting smarter over time, Huang still believes that there should always be a human backup, even in instances where a car is designed without a driver seat. To achieve this, Nvidia showcased its Holodeck during this year’s GTC keynote, allowing a remote driver to control a physical car in real-time through virtual reality.

“It’s teleportation,” Huang said, highlighting that this is possible through Nvidia’s early investments in virtual reality.

NVIDIA DRIVE—GTC 2018 Demonstration

During the demo, Tim, the driver, was located in a remote location. When he put on a pair of virtual reality glasses, he will felt like he’s in a physical car, enabling him to feel the car, and see the car’s controls and instrument panel. From this remote location and with the aid of his VR headset, he could take control of an autonomous vehicle, allowing him to drive the vehicle and park it.

It’s like what the military has been doing for a while — allowing drone operators to fly unmanned drones from remote location. But in Nvidia’s case, with the power of VR, the driver will feel like he is physically present in the cockpit. The company believes that simulation powered by its GPUs will eventually make autonomous cars nigh-infallible but, until them, the Holodeck can help humans watch over self-driving fleets.

Editors' Recommendations

Chuong Nguyen
Silicon Valley-based technology reporter and Giants baseball fan who splits his time between Northern California and Southern…
Ex-Apple employee pleads guilty to nabbing Apple Car secrets
The Apple logo is displayed at the Apple Store June 17, 2015 on Fifth Avenue in New York City

A former Apple employee on Monday pled guilty to the theft of trade secrets from the tech firm.

The material stolen by Xiaolang Zhang was linked to Apple’s work on its first-ever automobile, a project that’s been in and out of the headlines for years though never officially confirmed by the company.

Read more
A weird thing just happened with a fleet of autonomous cars
A passenger getting into a Cruise robotaxi.

In what must be one of the weirder stories linked to the development of autonomous vehicles, a fleet of Cruise self-driving cars gathered together at an intersection in San Francisco earlier this week, parked up, and blocked traffic for several hours. And to be clear: No, they weren't supposed to do that.

Some observers may have thought they were witnessing the start of the robot uprising, but the real reason for the mishap was more prosaic: An issue with the platform's software.

Read more
Officers confused as they pull over an empty self-driving car

In what appears to be the first incident of its kind, police officers recently pulled over a self-driving car with no one inside it.

The incident, which took place on a street in San Francisco earlier this month, was caught on video by a passing pedestrian. It shows several traffic cops pondering about how to handle the incident after stopping the vehicle for failing to have its front lights on while driving at night.

Read more