How Nvidia is helping autonomous cars simulate their way to safety

how nvidia is helping autonomous cars simulate their way to safety gtc path billions of miles

Imagine that you’re the driver of a four-door family sedan approaching a stop sign. When you reach the stop sign, you notice a bicyclist trying to cross the road. Through eye contact, facial expression and body language cues, the bicyclist negotiates their right of way with you. As a result, you decide to let the bicyclist cross the road first, before you proceed to cautiously enter the intersection.

In the autonomous driving world today, there would be no way to “tag” or categorize such an event, said Cognata CEO Danny Atsmon. Current methods allow you to visually identify the bicyclist, but training systems to recognize and understand complex negotiations on the road remain a challenge for the $10.3 trillion autonomous driving industry.

In fact, autonomous driving represents “the single hardest computing problem the world has ever encountered,” as NVIDIA CEO Jensen Huang admitted when he unveiled some of the world’s most powerful graphics processors during the GTC 2018 keynote in San Jose, California.

Bridging the Real and the Virtual

“The world drives 10 trillion miles per year,” Huang said in a pointed presentation — but Atsmon pointed out that self-driving cars only covered three million miles of roads last year. For self-driving vehicles to drive better, they must learn more, and that’s fundamentally the largest challenge faced by the industry. To train an autonomous driving system to have the competence of a human driver, computers would need to drive roughly 11 billion miles, Atsmon told us.

It’s the single hardest computing problem the world has ever encountered.

That figure is calculated based on the 1.09 fatalities per 100 million miles driven in 2015. “So, to say a machine could have as safe a performance as a human being with 95 percent of confidence, you would need to validate for 11 billion miles,” said Atsmon.

Aside from the time needed to reach that goal, there’s also the expense to consider. Right now, the cost per mile for operating an autonomous car is in the hundreds of dollars — accounting for engineering time, data collection and tagging, insurance costs, and the time of a driver to sit in the cockpit of a car. Multiply that by the 11-billion-mile benchmark, and the massive expensive associated with training autonomous cars becomes clear.

Validation is key, and recent accidents involving autonomous vehicles show that incomplete data tests and training scenarios can prove fatal. In one less extreme example, a self-driving shuttle in Las Vegas was navigating at around 0.6 miles per hour, but it crashed into a truck (Jeff Zurschmeide, a freelancer contributor to Digital Trends, was there when it happened). No one was injured, but the puzzling scenario happened because the truck was pulling forward, then backing up as it tried to park. The cause for the crash, according to Atsmon, is that the shuttle wasn’t validated for this type of situation, and it didn’t know what to do — so it proceeded forward slowly and crashed.

Better Simulation for Deeper Learning

The industry’s current solution to bridging the 11-billion-mile gap for autonomous systems to reach human driving competency is to develop simulations to allow cars to learn faster by combining deep learning with a virtual environment.

“Simulation is the path to billions of miles,” Huang said at GTC. Late last year, Alphabet-owned Waymo unveiled Carcraft, its approach to learning by simulation.

Cognata is using the latest advancements in graphics and sensor hardware to create more life-like and realistic models of the world for autonomous cars to learn from. For the computing brains of a self-driving car, it’s like entering a video game modeled on the real world, and that could lead to more realistic driving scenarios to test and validate car driving data. The company has recently mapped out select cities, like San Francisco, using data from GIS — high definition cameras and sophisticated computer algorithms that runs over satellite and street view imagery, resulting in a photo-realistic scene.

Simulation is the path to billions of miles.

To further improve simulations, Nvidia, and some of its partners, are using data from the sensors of autonomous vehicles to build higher definition maps. When autonomous vehicles hit the road, these machines will not only rely on the data that is available through training, but also contribute to data collection by sharing the data that it has captured from its LIDAR, IR, radar, and camera arrays.

When this newly captured data is combined through deep learning with existing low-quality data sets, it will make streets and roads look more photo-realistic. Cognata claims that its algorithms can process the data in a way to bring out details in shadows and highlights, much like an HDR photo from your smartphone’s camera, to create a high-quality scene.

While simulation is an excellent tool, Atsmon noted it has its own flaws. It’s too simple, and for autonomous driving to be realistic, it must learn from edge cases. Cognata claims that it only takes a few clicks to program in an edge case to validate autonomous vehicles for more unusual driving scenarios. Companies building autonomous vehicles will have to be diligent in their search for edge cases that can trick self-driving cars, and creative in crafting solutions for them.

When Self-Driving Fails

Safety is so paramount to autonomous vehicles that Nvidia considers it the single most important thing for the industry. When things fail, fatalities can and do occur, as was recently proven when an autonomous Uber struck and killed a pedestrian in Arizona.

“I can assure you that [Uber is] equally crushed at what happened.”

When questioned in a press meeting about the Uber crash — Uber is a partner of Nvidia — Huang deferred to the ride-sharing company for comments, saying that “we should give Uber a chance to understand what has happened and to explain what has happened.”

“I can assure you that [Uber is] equally crushed at what happened,” Huang added.

Because Nvidia develops an end-to-end solution for autonomous driving, different partners — from Uber to Toyota and Mercedes Benz — may utilize all or some parts of the system. “There are some 370 companies around the world who are using our technologies in some way.” At the show, Nvidia also announced Orin, the next-generation computer of its DRIVE platform.


Humans as a Backup

While self-driving cars are getting smarter over time, Huang still believes that there should always be a human backup, even in instances where a car is designed without a driver seat. To achieve this, Nvidia showcased its Holodeck during this year’s GTC keynote, allowing a remote driver to control a physical car in real-time through virtual reality.

“It’s teleportation,” Huang said, highlighting that this is possible through Nvidia’s early investments in virtual reality.

During the demo, Tim, the driver, was located in a remote location. When he put on a pair of virtual reality glasses, he will felt like he’s in a physical car, enabling him to feel the car, and see the car’s controls and instrument panel. From this remote location and with the aid of his VR headset, he could take control of an autonomous vehicle, allowing him to drive the vehicle and park it.

It’s like what the military has been doing for a while — allowing drone operators to fly unmanned drones from remote location. But in Nvidia’s case, with the power of VR, the driver will feel like he is physically present in the cockpit. The company believes that simulation powered by its GPUs will eventually make autonomous cars nigh-infallible but, until them, the Holodeck can help humans watch over self-driving fleets.


Many adults believe fully self-driving cars are already traversing U.S. highways

The American Automobile Association tested cars with features such as lane-keeping and adaptive cruise control and found them lacking in real-world conditions. Forty percent of surveyed U.S. adults think self-driving cars exist now.

Ford teams up with Walmart to study consumer response to autonomous delivery

Last week it was Ford and VW, and this week Ford and Walmart are signaling a desire to work together on autonomous vehicles solutions. Ford and the giant retailer will study consumer reactions to self-driving delivery vehicles.

Waymo will launch its commercial autonomous ridesharing service in December

Waymo will launch a commercial ridesharing service using self-driving cars in December, according to a new report. As previously discussed by Waymo, the service will operate in specific areas around Phoenix, Arizona.

Study suggests autonomous cars could become red-light districts on wheels

Fully autonomous cars can change the way we commute, but they can also have a far-reaching impact on the tourism industry. Two researchers published a study that outlines how self-driving technology could create a new dimension in tourism.

Land Rover shows its artsy side by previewing the 2020 Range Rover Evoque

Land Rover made life-sized wire sculptures to preview the 2020 Range Rover Evoque. The all-new SUV will make its debut during a private event held in London on November 22, and deliveries will begin in 2019.

These headlights have 4 color settings to help save lives under all conditions

Boslla headlights are an easy-to-install solution to achieve all-weather lighting for your vehicle. After a quick ten-minute install, these lights have up to four settings to get you through anything.

Car parts maker ZF is using drones to deliver components to its factories

ZF recently became the first entity in Germany to receive approval to use drones to deliver spare parts, and the company now uses them to deliver parts from its central warehouses to its workshops.

The DBX SUV will go where no Aston Martin has gone before

When it launches in 2019, the Aston Martin DBX will be the British automaker's first SUV. In the meantime, camouflaged DBX prototypes will undergo strenuous testing around the world.

Our favorite fuel-efficient cars are as frugal as they are fun

You don't need to opt for a hybrid or an all-electric ride in order to achieve good fuel economy. These vehicles pack both performance and style, whether you're in the market for a luxury sedan or a game-changing pickup truck.

Out of juice? Learn how to jump-start a car with this quick guide

Jumping a car is a simple procedure, but not everyone knows how to properly do so. To make things easier, we've put together a quick-hit guide on how to fire up your vehicle using jumper cables and a second power source.

Born to run (forever): The most reliable cars you can buy right now

We all dread the thought of our car turning into a money pit, but choosing a dependable vehicle from the start can help us rack up countless care-free miles. Here, we've rounded up some of the most reliable cars available.

Prep your car for the coming snow and sleet with these cold weather tips

Driving in the winter, whether downtown or across the country, is rarely easy. Luckily, we've put together a quick rundown of a few things you should do to winterize your car before the snow officially hits.
DT Daily

DT Daily: Waymo’s driverless cars, ‘Fallout 76’ tips, and Racella

In today's episode of DT Daily, we discuss Waymo's foray into the ridesharing sector, along with various tips for making the most of the recently launched Fallout 76. We also sit down with singer Racella to chat about her new EP, Waves.

Want to keep connected on the road? Here are 5 ways to add Bluetooth to your car

The best way to make an old ride feel young again is to bring it up speed with the 21st century. Here's how to properly add Bluetooth to your vehicle, via independent kits, vehicle adapters, or aftermarket head units.