Skip to main content

School’s in session — Nvidia’s driverless system learns by watching

nvidia gpu driverless car project 100658383 primary idge 900x599
Nvidia's GPU-based driverless system learns by watching human drivers Nvidia
How do you train a car to drive itself? Let it watch real drivers. Engineers from graphics processing unit (GPU) company Nvidia designed a system that learned how to drive after watching humans drive for a total of 72 hours, as reported by NetworkWorld. The likely conclusion of the system’s success is that driverless cars are coming faster than most of us expected.

The details of how Nvidia trained two test cars are in a file titled End to End Learning for Self-Driving Cars. The basics are as follows: Nvidia used three cameras and two Nvidia DRIVE PX computers to watch humans drive cars for 72 hours.  The car training was conducted in several states, in varied weather conditions, day and night, and on a wide variety of road types and conditions. The cameras captured massive amounts data in 3D, which was tracked and stored by the on-board GPUs. That data was then analyzed and broken into learning steps with Torch 7, a machine learning system. The output? A trained system for driving cars without driver intervention.

In subsequent tests using the trained system in cars on test roads and public roads (including highways), the system achieved 98 to 100 percent autonomy. When a driver intervenes, the system continues to learn.

Only one camera and one computer are needed in a car based on Nvidia’s GPU system. When the time comes that you can buy a car with this system, it will arrive ready to drive itself — you won’t need to give it more training runs, as told to Digital Trends. While the car can drive itself, when it detects (via the camera) something new (as in something it hasn’t “seen” before), it alerts the driver to take over and then goes into learning mode. Whatever it learns is later transmitted to the cloud and that information is incorporated into the next software update so the learning can benefit all cars using Nvidia’s self-driving system. You can watch Nvidia’s GPU-based car training and performance here.

When speaking with Digital Trends, a spokesperson from Nvidia used the example of encountering a moose. The moose example works because chancing upon a moose in the road is rare in most parts of the U.S. and apparently has never happened in their car training. So if I was driving a car with the Nvidia self-driving system and a moose was on or beside the road, the system would alert me to take over. The onboard system would watch what I did and record changes to steering and other systems like brakes and acceleration. My reactions would be transmitted to the cloud. After the next software update, if another person encountered a moose, that car would know how to react based on my reactions.

This learn-by-watching method of training driverless vehicles is both more realistic and inclusive than chains of rules written by programmers based on highly varied elements like road surface, lane markers, lights, and traffic conditions. The GPU-based system does the data-gathering and the machine learning system creates the rules.

Digital Trends was informed that Nvidia is now in conversations with more than 80 major OEMs (original equipment manufacturers) and research institutions. The one manufacturer Nvidia was able speak about publicly is Volvo. Next year Volvo’s Drive Me project in Gothenburg, Sweden will use 100 Volvo XC90s equipped with the Nvidia system to observe how the system works on “a defined set of roads” in the Swedish city.

Driverless cars are coming and Nvidia’s system looks to be on pace to speed up the transition.

Editors' Recommendations

Bruce Brown
Digital Trends Contributing Editor Bruce Brown is a member of the Smart Homes and Commerce teams. Bruce uses smart devices…
It looks like no one is buying Nvidia’s RTX 4070
The RTX 4070 graphics card on a pink background.

Despite some positive reviews, including our own RTX 4070 review, Nvidia's latest graphics card is reportedly seeing poor sales. Only one week after launching, it's being beat by last-gen GPUs on bestseller charts, and reports suggest Nvidia could pause production to keep the price stable.

Wccftech reported that an update posted on the Chinese Board Channel forums says Nvidia could temporarily halt production in order to "maintain a stable price system." It's hard to say if Nvidia will actually pause production or not. Inventory and sales numbers are rarely shared, so it's important to handle a forum post like this with some skepticism.

Read more
Watch out — Diablo IV might break your Nvidia GPU
Screenshot from the Diablo IV trailer.

Following last weekend's Diablo IV open beta, reports came flooding in saying that the game might be breaking Nvidia graphics cards. This means various crashes and subpar performance, as well as even outright dead GPUs.

Fortunately, not all cards are affected, and there are steps you can take to protect yourself. Here's what we know.

Read more
Cruise’s robotaxis have driven 1 million miles fully driverless
A driverless Cruise car in San Francisco.

General Motors-backed Cruise revealed this week that its fully driverless cars have now traveled more than a million miles, mostly on the streets of San Francisco.

The achievement comes just 15 months after the company’s first fully driverless ride, during which time it also launched San Francisco’s first paid driverless robotaxi service.

Read more