Here’s how Bosch teaches cars to see using artificial intelligence

A car needs to be able to see its environment before it can drive itself. Don’t let pareidolia fool you; its headlights aren’t eyes. They are made out of metal, glass, and plastic parts, and they rely on an enormous amount of computing power to remain open. Without them, the car’s brain isn’t able to make the right decision at the right time.

AI by Bosch is the brains behind many self-driving car platforms, and to see how it works — and how a car sees the world around it — the company gave us a chance to briefly explore the German countryside in one of its prototypes. It turns out Andy Warhol and the future of mobility have more in common than you might think.

Virtual 20/20 vision

The prototype we’re riding shotgun in looks like a garden-variety BMW 3 Series station wagon when you see it from the outside. There are tens of thousands of them on German roads, so what makes this one special? After settling into the leather-upholstered passenger seat we notice it’s decked out with cameras, sensors, and radars that are attached to the windshield, though we’re told only the monocular camera is turned on during our trip. There is also an additional panel on the center console with various input ports, and a tablet mounted on the dashboard.

The technology is relatively simple – at least on paper. The windshield-mounted camera records footage and sends it to a PC stuffed in the Bimmer’s trunk. The information goes through a graphic processing unit (GPU) manufactured by Nvidia before traveling to the car’s on-board brain. The tablet on the dashboard is only there for demonstration purposes.

Our prototype splits up the outside world into 19 categories and each one is identified by a different color.

Artificial intelligence helps our prototype split up the outside world into 19 categories. Each one is identified by a different color, which creates a Pop Art-like view of what’s ahead. It knows the difference between a street and a sidewalk, and it can identify various objects such as traffic signs, traffic lights, pedestrians, and different types of vehicles including cars, trucks, and bicycles. Much like a human driver, the car recognizes which objects are safe to drive over and which ones it needs to brake for.

It’s smart enough to identify what’s ahead with surprising speed and accuracy; it’s been taught a street sign is not a tree or a small child riding a skateboard. It’s chilling to think about. We’re riding in a BMW station wagon that knows almost as much about driving in a city as its two occupants.

Back to school

The prototype learned everything it knows from members of Bosch’s research and development department.

“We have an offline training process,” explained research engineer Dimitrios Bariamis in an interview with Digital Trends. “We give the car images that we annotate, so we say ‘in this part of the image there is a pedestrian, this part of the image is a street,’ and so on. Then we get that into the car, and we give it the image from the camera which is processed according to the parameters that have been previously learned. The system knows that this part of the image is a street because it looks like the street it saw during the training process,” he adds.

Bariamis and his team have fed the system thousands of screen shots from on-board video footage taken in German cities like Munich, Frankfurt, and Stuttgart. They also sourced images from Daimler’s Cityscapes Dataset, which breaks the world down into the exact same 19 categories. These annotated images help the car learn as it moves along, even if it’s traveling in a town it’s never been to before. “Artificial intelligence generalizes the unknown,” Bariamis tells us.

The software classifies the world around it even in a heavy rain storm, but it hasn’t been tested in the snow yet. Bariamis is optimistic, and he doesn’t think snow will impair the car’s vision. Right now, the only limitations his team has identified are linked to what the car has and hasn’t seen, and hardware issues. For example, the system has never “seen” a highway yet, so it might not be able to identify a toll booth. It also goes without saying that the car loses its eyesight if something – e.g., the viscous contents of an avian digestive system – suddenly covers up the camera.

The project is the work of Bosch’s forward-thinking research and development arm. Where it goes next depends entirely on the company’s corporate arm and its clients.

Bariamis told us the technology can be integrated into relatively basic driver-assistance features like adaptive cruise control, state-of-the-art semi-autonomous software, and even a fully-autonomous car. Crucially, it can be modified for various uses. The software we experienced in Germany sees the world in 19 colors, but it’s possible to either add more categories when more detailed information is required, or delete a few of them if they’re not needed.

The Warhol-esque view of the world showcased by Bosch’s BMW-based prototype is what will make the advent of robot cars possible in the years to come. It’s an integral part of the technology package that will help the automotive industry transition from building cars to manufacturing intelligent cars.

Cars

BMW’s 2020 M340i will show off mix of tech and luxury at the Los Angeles show

The 2020 3 Series represents the seventh generation of BMW's bread-and-butter luxury sedan. Where previous generations emphasized the driving experience, the 2020 3 Series focuses more on tech than ever before.
Home Theater

Put your home theater to the test with these spectacular Blu-ray releases

What's the point of having all of that awesome home theater gear if you can't breed a little jealousy in your friends and family? We've put together this list of fantastic Blu-rays that have the goods to drop a few jaws.
Cars

Prep your car for the coming snow and sleet with these cold weather tips

Driving in the winter, whether downtown or across the country, is rarely easy. Luckily, we've put together a quick rundown of a few things you should do to winterize your car before the snow officially hits.
Deals

The best iPhone deals for November 2018

Apple devices can get expensive, but if you just can't live without iOS, don't despair: We've curated an up-to-date list of all of the absolute best iPhone deals available for November 2018.
Cars

Our favorite fuel-efficient cars are as frugal as they are fun

You don't need to opt for a hybrid or an all-electric ride in order to achieve good fuel economy. These vehicles pack both performance and style, whether you're in the market for a luxury sedan or a game-changing pickup truck.
Cars

Out of juice? Learn how to jump-start a car with this quick guide

Jumping a car is a simple procedure, but not everyone knows how to properly do so. To make things easier, we've put together a quick-hit guide on how to fire up your vehicle using jumper cables and a second power source.
Cars

Many adults believe fully self-driving cars are already traversing U.S. highways

The American Automobile Association tested cars with features such as lane-keeping and adaptive cruise control and found them lacking in real-world conditions. Forty percent of surveyed U.S. adults think self-driving cars exist now.
Cars

Born to run (forever): The most reliable cars you can buy right now

We all dread the thought of our car turning into a money pit, but choosing a dependable vehicle from the start can help us rack up countless care-free miles. Here, we've rounded up some of the most reliable cars available.
Digital Trends Live

DT Daily: Waymo’s driverless cars, ‘Fallout 76’ tips, and Racella

In today's episode of DT Daily, we discuss Waymo's foray into the ridesharing sector, along with various tips for making the most of the recently launched Fallout 76. We also sit down with singer Racella to chat about her new EP, Waves.
Cars

Want to keep connected on the road? Here are 5 ways to add Bluetooth to your car

The best way to make an old ride feel young again is to bring it up speed with the 21st century. Here's how to properly add Bluetooth to your vehicle, via independent kits, vehicle adapters, or aftermarket head units.
Cars

Jeep’s outdoorsy Gladiator pickup truck bares it all ahead of schedule

Jeep will introduce the long-promised Wrangler-based pickup truck at the 2018 Los Angeles Auto Show. Named Gladiator, the model was designed to conquer the great outdoors, not for the construction site.
Cars

The hamster-friendly 2020 Kia Soul will rock out at the Los Angeles Auto Show

Kia has released a teaser image to preview the next Soul. Scheduled to make its debut at the 2018 Los Angeles Auto Show, the 2020 Soul will keep the outgoing model's boxy proportions but it will wear a sharper design.
Cars

2020 Toyota Corolla sedan aims to offer sharper handling, better tech

The 2020 Toyota Corolla sedan gets the same upgrades as the recently-introduced Corolla hatchback, including a firmer foundation, new engine, and more tech features. Will that be enough to keep the long-lived Corolla nameplate relevant?
Cars

Startup Rivian exits stealth mode with a bold promise to electrify off-roaders

Electric car startup Rivian has finally turned off stealth mode and provided details about what it's been working on since 2009. It will build battery-powered off-roaders instead of taking on Tesla and others in the luxury EV segment.