How Nvidia is helping autonomous cars simulate their way to safety

how nvidia is helping autonomous cars simulate their way to safety gtc path billions of miles

Imagine that you’re the driver of a four-door family sedan approaching a stop sign. When you reach the stop sign, you notice a bicyclist trying to cross the road. Through eye contact, facial expression and body language cues, the bicyclist negotiates their right of way with you. As a result, you decide to let the bicyclist cross the road first, before you proceed to cautiously enter the intersection.

In the autonomous driving world today, there would be no way to “tag” or categorize such an event, said Cognata CEO Danny Atsmon. Current methods allow you to visually identify the bicyclist, but training systems to recognize and understand complex negotiations on the road remain a challenge for the $10.3 trillion autonomous driving industry.

In fact, autonomous driving represents “the single hardest computing problem the world has ever encountered,” as NVIDIA CEO Jensen Huang admitted when he unveiled some of the world’s most powerful graphics processors during the GTC 2018 keynote in San Jose, California.

Bridging the Real and the Virtual

“The world drives 10 trillion miles per year,” Huang said in a pointed presentation — but Atsmon pointed out that self-driving cars only covered three million miles of roads last year. For self-driving vehicles to drive better, they must learn more, and that’s fundamentally the largest challenge faced by the industry. To train an autonomous driving system to have the competence of a human driver, computers would need to drive roughly 11 billion miles, Atsmon told us.

It’s the single hardest computing problem the world has ever encountered.

That figure is calculated based on the 1.09 fatalities per 100 million miles driven in 2015. “So, to say a machine could have as safe a performance as a human being with 95 percent of confidence, you would need to validate for 11 billion miles,” said Atsmon.

Aside from the time needed to reach that goal, there’s also the expense to consider. Right now, the cost per mile for operating an autonomous car is in the hundreds of dollars — accounting for engineering time, data collection and tagging, insurance costs, and the time of a driver to sit in the cockpit of a car. Multiply that by the 11-billion-mile benchmark, and the massive expensive associated with training autonomous cars becomes clear.

Validation is key, and recent accidents involving autonomous vehicles show that incomplete data tests and training scenarios can prove fatal. In one less extreme example, a self-driving shuttle in Las Vegas was navigating at around 0.6 miles per hour, but it crashed into a truck (Jeff Zurschmeide, a freelancer contributor to Digital Trends, was there when it happened). No one was injured, but the puzzling scenario happened because the truck was pulling forward, then backing up as it tried to park. The cause for the crash, according to Atsmon, is that the shuttle wasn’t validated for this type of situation, and it didn’t know what to do — so it proceeded forward slowly and crashed.

Better Simulation for Deeper Learning

The industry’s current solution to bridging the 11-billion-mile gap for autonomous systems to reach human driving competency is to develop simulations to allow cars to learn faster by combining deep learning with a virtual environment.

“Simulation is the path to billions of miles,” Huang said at GTC. Late last year, Alphabet-owned Waymo unveiled Carcraft, its approach to learning by simulation.

Cognata is using the latest advancements in graphics and sensor hardware to create more life-like and realistic models of the world for autonomous cars to learn from. For the computing brains of a self-driving car, it’s like entering a video game modeled on the real world, and that could lead to more realistic driving scenarios to test and validate car driving data. The company has recently mapped out select cities, like San Francisco, using data from GIS — high definition cameras and sophisticated computer algorithms that runs over satellite and street view imagery, resulting in a photo-realistic scene.

Simulation is the path to billions of miles.

To further improve simulations, Nvidia, and some of its partners, are using data from the sensors of autonomous vehicles to build higher definition maps. When autonomous vehicles hit the road, these machines will not only rely on the data that is available through training, but also contribute to data collection by sharing the data that it has captured from its LIDAR, IR, radar, and camera arrays.

When this newly captured data is combined through deep learning with existing low-quality data sets, it will make streets and roads look more photo-realistic. Cognata claims that its algorithms can process the data in a way to bring out details in shadows and highlights, much like an HDR photo from your smartphone’s camera, to create a high-quality scene.

While simulation is an excellent tool, Atsmon noted it has its own flaws. It’s too simple, and for autonomous driving to be realistic, it must learn from edge cases. Cognata claims that it only takes a few clicks to program in an edge case to validate autonomous vehicles for more unusual driving scenarios. Companies building autonomous vehicles will have to be diligent in their search for edge cases that can trick self-driving cars, and creative in crafting solutions for them.

When Self-Driving Fails

Safety is so paramount to autonomous vehicles that Nvidia considers it the single most important thing for the industry. When things fail, fatalities can and do occur, as was recently proven when an autonomous Uber struck and killed a pedestrian in Arizona.

“I can assure you that [Uber is] equally crushed at what happened.”

When questioned in a press meeting about the Uber crash — Uber is a partner of Nvidia — Huang deferred to the ride-sharing company for comments, saying that “we should give Uber a chance to understand what has happened and to explain what has happened.”

“I can assure you that [Uber is] equally crushed at what happened,” Huang added.

Because Nvidia develops an end-to-end solution for autonomous driving, different partners — from Uber to Toyota and Mercedes Benz — may utilize all or some parts of the system. “There are some 370 companies around the world who are using our technologies in some way.” At the show, Nvidia also announced Orin, the next-generation computer of its DRIVE platform.


Humans as a Backup

While self-driving cars are getting smarter over time, Huang still believes that there should always be a human backup, even in instances where a car is designed without a driver seat. To achieve this, Nvidia showcased its Holodeck during this year’s GTC keynote, allowing a remote driver to control a physical car in real-time through virtual reality.

“It’s teleportation,” Huang said, highlighting that this is possible through Nvidia’s early investments in virtual reality.

During the demo, Tim, the driver, was located in a remote location. When he put on a pair of virtual reality glasses, he will felt like he’s in a physical car, enabling him to feel the car, and see the car’s controls and instrument panel. From this remote location and with the aid of his VR headset, he could take control of an autonomous vehicle, allowing him to drive the vehicle and park it.

It’s like what the military has been doing for a while — allowing drone operators to fly unmanned drones from remote location. But in Nvidia’s case, with the power of VR, the driver will feel like he is physically present in the cockpit. The company believes that simulation powered by its GPUs will eventually make autonomous cars nigh-infallible but, until them, the Holodeck can help humans watch over self-driving fleets.


Driving Daimler’s 40-ton eCascadia big rig isn’t just fun, it’s electrifying

Daimler Trucks brought its all-electric eCascadia semi-truck to the 2019 CES, and invited us to take the wheel. What does it feel like to drive one? Simply electrifying, of course.
Emerging Tech

Too buzzed to drive? Don’t worry — this autonomous car-bar will drive to you

It might just be the best or worst idea that we've ever heard: A self-driving robot bartender you can summon with an app, which promises to mix you the perfect drink wherever you happen to be.

How to share an external hard drive between Mac and Windows

Compatibility issues between Microsoft Windows and Apple MacOS may have diminished sharply over the years, but that doesn't mean they've completely disappeared. Here's how to make an external drive work between both operating systems.

Nissan IMs concept teases a future long-range, autonomous electric car

Debuting at the 2019 Detroit Auto Show, the Nissan IMs is an electric car with a 380-mile range, autonomous-driving capability, and a backseat designed for being chauffeured. Too bad it's just a concept car.

Fast and Furious fans get revved up: Toyota’s Supra sports car is back

The 2020 Toyota Supra made its long-awaited debut at the 2019 Detroit Auto Show. The resurrected sports car, famous for a role in The Fast and the Furious, goes on sale in the U.S. this summer.

Muscle cars, trucks, and EVs roared into the subdued 2019 Detroit Auto Show

The 2019 Detroit Auto Show was the quietest edition of the event in recent memory, but that doesn't mean nothing significant happened inside the Cobo Center. Here are the new cars and concepts we saw at the show.

Big tech, bigger grille: BMW updates its 7 Series flagship for 2020

The BMW 7 Series will enter the 2020 model year with a host of updates inside, outside, and under the sheet metal. The new-look nose with a jumbo grille hides updated engines, while passengers benefit from smart tech features.
Emerging Tech

Ford’s sweaty robot bottom can simulate 10 years of seat use in mere days

Ford has developed 'Robutt,' a sweaty robot bottom that's designed to simulate the effects of having a pair of human buttocks sitting on its car seats for thousands of hours. Check it out.

In McLaren’s 600LT Spider, the engine is the only sound system you’ll need

The McLaren 600LT Spider is the inevitable convertible version of the 600LT coupe, itself a lighter, more powerful version of the McLaren 570S. The 600LT Spider boasts a 592-horsepower, twin-turbo V8, and a loud exhaust system to hear it…

Robomart’s self-driving grocery store is like Amazon Go on wheels

Robomart's driverless vehicle is like an Amazon Go store on wheels, with sensors tracking what you grab from the shelves. If you don't want to shop online or visit the grocery store yourself, Robomart will bring the store to you.

Ford has a plan to future-proof the hot-selling F-150 pickup truck

Worried about the threat of rising gas prices, Ford will add the F-150 to its growing portfolio of electrified vehicles. It is currently developing a hybrid F-150, and it will release an electric version of the next-generation truck.

Ford’s Mustang-inspired electric crossover will spawn a Lincoln luxury version

Lincoln will get its own version of parent Ford's first mass-market, long-range electric vehicle. While Ford's version will have styling inspired by the Mustang, Lincoln will take a more traditional approach.
Home Theater

Spotify adds simplified Car View mode for Android users

What was once just a test is now a reality: Spotify is rolling out a new, simplified in-car user interface for all Android users called Car View, which automatically engages when the app detects a car Bluetooth connection.

Boutique carmaker Karma Automotive, legendary design firm Pininfarina team up

Karma Automotive is partnering with legendary Italian design firm Pininfarina on future luxury cars. The first product of that partnership will appear later this year, Karma said, without offering other details.