Skip to main content

School’s in session — Nvidia’s driverless system learns by watching

nvidia gpu driverless car project 100658383 primary idge 900x599
Nvidia's GPU-based driverless system learns by watching human drivers Nvidia
How do you train a car to drive itself? Let it watch real drivers. Engineers from graphics processing unit (GPU) company Nvidia designed a system that learned how to drive after watching humans drive for a total of 72 hours, as reported by NetworkWorld. The likely conclusion of the system’s success is that driverless cars are coming faster than most of us expected.

The details of how Nvidia trained two test cars are in a file titled End to End Learning for Self-Driving Cars. The basics are as follows: Nvidia used three cameras and two Nvidia DRIVE PX computers to watch humans drive cars for 72 hours.  The car training was conducted in several states, in varied weather conditions, day and night, and on a wide variety of road types and conditions. The cameras captured massive amounts data in 3D, which was tracked and stored by the on-board GPUs. That data was then analyzed and broken into learning steps with Torch 7, a machine learning system. The output? A trained system for driving cars without driver intervention.

In subsequent tests using the trained system in cars on test roads and public roads (including highways), the system achieved 98 to 100 percent autonomy. When a driver intervenes, the system continues to learn.

Get your weekly teardown of the tech behind PC gaming
Check your inbox!

Only one camera and one computer are needed in a car based on Nvidia’s GPU system. When the time comes that you can buy a car with this system, it will arrive ready to drive itself — you won’t need to give it more training runs, as told to Digital Trends. While the car can drive itself, when it detects (via the camera) something new (as in something it hasn’t “seen” before), it alerts the driver to take over and then goes into learning mode. Whatever it learns is later transmitted to the cloud and that information is incorporated into the next software update so the learning can benefit all cars using Nvidia’s self-driving system. You can watch Nvidia’s GPU-based car training and performance here.

When speaking with Digital Trends, a spokesperson from Nvidia used the example of encountering a moose. The moose example works because chancing upon a moose in the road is rare in most parts of the U.S. and apparently has never happened in their car training. So if I was driving a car with the Nvidia self-driving system and a moose was on or beside the road, the system would alert me to take over. The onboard system would watch what I did and record changes to steering and other systems like brakes and acceleration. My reactions would be transmitted to the cloud. After the next software update, if another person encountered a moose, that car would know how to react based on my reactions.

This learn-by-watching method of training driverless vehicles is both more realistic and inclusive than chains of rules written by programmers based on highly varied elements like road surface, lane markers, lights, and traffic conditions. The GPU-based system does the data-gathering and the machine learning system creates the rules.

Digital Trends was informed that Nvidia is now in conversations with more than 80 major OEMs (original equipment manufacturers) and research institutions. The one manufacturer Nvidia was able speak about publicly is Volvo. Next year Volvo’s Drive Me project in Gothenburg, Sweden will use 100 Volvo XC90s equipped with the Nvidia system to observe how the system works on “a defined set of roads” in the Swedish city.

Driverless cars are coming and Nvidia’s system looks to be on pace to speed up the transition.

Editors' Recommendations

Bruce Brown
Digital Trends Contributing Editor Bruce Brown is a member of the Smart Homes and Commerce teams. Bruce uses smart devices…
This underrated Nvidia GPU is still the one to buy
Nvidia's RTX 4070 graphics cards over a pink background.

There hasn't been a lot of positive reception to Nvidia's graphics cards this generation. There's no doubt that Nvidia's RTX 40-series GPUs are among the best graphics cards you can buy, but with disappointing generational improvements, higher pricing, and reliance on features like DLSS 3, it's easy to write off a new graphics card when Nvidia releases one.

But there's one Goldilocks graphics card from Nvidia's current generation that's the right choice for a lot of PC gamers. It delivers all of the best parts of Nvidia's cutting-edge generation and ignores the insane pricing and disappointing generational uplifts. And now, months after launch, prices have dropped, making it an even better value. That GPU is Nvidia's RTX 4070.
Not a warm reception

Read more
Nvidia doesn’t want you to know about its controversial new GPU
Logo on the RTX 4060 Ti graphics card.

Confirming previous rumors, Nvidia released its RTX 4060 Ti on Tuesday without much fanfare. Most of the best graphics cards release with a bang, but the updated 16GB RTX 4060 Ti released with barely a whimper. Nvidia is putting more weight behind Portal: Prelude RTX, on the same day it's launching a new graphics card. Weird.

This is the 16GB variant that Nvidia promised when the original 8GB RTX 4060 Ti was released in June, and although it has technically been released, you can't buy it.

Read more
Should you buy Nvidia’s RTX 4060 or RTX 4060 Ti?
Two RTX 4060 graphics cards stacked on top of each other.

Nvidia makes some of the best graphics cards you can buy, but two have been the topic of much debate over the past several weeks: the RTX 4060 and RTX 4060 Ti. These two GPUs are meant to hit the mainstream market for Team Green, though they struggle to hit that mark to varying degrees.

We put both on the test bench to see if you should buy the RTX 4060 or the RTX 4060 Ti. If you want a more in-depth look, make sure to read our RTX 4060 review and RTX 4060 Ti review.
Specs

Read more