Skip to main content

Could thermal-imaging sensors help make self-driving cars safer?

FLIR Systems

Recently we published a story about the history of notable car accidents involving self-driving and semi-self-driving vehicles. While a significant number of these do involve human error, there are still multiple examples of crashes in which the vehicle was unable to properly read its surroundings. Thermal imaging company FLIR Systems thinks one way to make autonomous vehicles safer would be to give them the ability to use thermal reading tech to better deal with challenging lighting and weather conditions.

Recommended Videos

To help with adoption of these sensors, the company has made available an open-source dataset of 10,000 labelled infrared light images showcasing how pedestrians, animals, bicycles and other vehicles can be classified using the tech in difficult conditions, ranging from total darkness to fog and smoke to haze and glare from the sun. Using thermal cameras, combined with the dataset, it’s possible to recognize objects more than 200 meters away, the equivalent of 4 times the distance of typical car headlights.

“Navigation is limited by the perception accuracy provided by the vehicle sensor suite,” Mike Walters, vice president of micro-camera product management at FLIR Systems, told Digital Trends. “Visible cameras do not work well in a number of challenging lighting situations such as absolute darkness, driving into the sun, and in many types of fog — while thermal cameras, which principally see heat and not visible light, are unaffected by these adverse conditions. Therefore, thermal cameras provide more accurate situational awareness and perception under these conditions, which, in turn, improves safety and navigation.”

At present, thermal cameras are found in a number of luxury passenger cars. Some high-end automakers, such as Porsche and BMW, already fit vehicles with thermal imaging sensors made by FLIR. However, they are not part of the standard suite of sensors helping power today’s most prominent self-driving vehicles. That may sound like an obvious oversight, but current self-driving cars don’t rely on only one imaging system by which to see.

The self-driving cars being tested on the road right now figure out their surroundings using a combination of regular cameras, ultrasound, radar, lidar and more. Information from all of these sensors helps inform the decisions the car makes. In some cases, this array of sensors may lead to redundancy — but few people, either passengers or pedestrians, are ever going to complain about being too safe.

As a result of this, it’s difficult to calculate how much of an improvement using technology like FLIR’s thermal long-wave infrared (LWIR) cameras could offer. Still, hopefully datasets such as the one newly released this week will give carmakers the opportunity to more easily build the algorithms that will let them find out.

Luke Dormehl
Former Digital Trends Contributor
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Tesla posts exaggerate self-driving capacity, safety regulators say
Beta of Tesla's FSD in a car.

The National Highway Traffic Safety Administration (NHTSA) is concerned that Tesla’s use of social media and its website makes false promises about the automaker’s full-self driving (FSD) software.
The warning dates back from May, but was made public in an email to Tesla released on November 8.
The NHTSA opened an investigation in October into 2.4 million Tesla vehicles equipped with the FSD software, following three reported collisions and a fatal crash. The investigation centers on FSD’s ability to perform in “relatively common” reduced visibility conditions, such as sun glare, fog, and airborne dust.
In these instances, it appears that “the driver may not be aware that he or she is responsible” to make appropriate operational selections, or “fully understand” the nuances of the system, NHTSA said.
Meanwhile, “Tesla’s X (Twitter) account has reposted or endorsed postings that exhibit disengaged driver behavior,” Gregory Magno, the NHTSA’s vehicle defects chief investigator, wrote to Tesla in an email.
The postings, which included reposted YouTube videos, may encourage viewers to see FSD-supervised as a “Robotaxi” instead of a partially automated, driver-assist system that requires “persistent attention and intermittent intervention by the driver,” Magno said.
In one of a number of Tesla posts on X, the social media platform owned by Tesla CEO Elon Musk, a driver was seen using FSD to reach a hospital while undergoing a heart attack. In another post, a driver said he had used FSD for a 50-minute ride home. Meanwhile, third-party comments on the posts promoted the advantages of using FSD while under the influence of alcohol or when tired, NHTSA said.
Tesla’s official website also promotes conflicting messaging on the capabilities of the FSD software, the regulator said.
NHTSA has requested that Tesla revisit its communications to ensure its messaging remains consistent with FSD’s approved instructions, namely that the software provides only a driver assist/support system requiring drivers to remain vigilant and maintain constant readiness to intervene in driving.
Tesla last month unveiled the Cybercab, an autonomous-driving EV with no steering wheel or pedals. The vehicle has been promoted as a robotaxi, a self-driving vehicle operated as part of a ride-paying service, such as the one already offered by Alphabet-owned Waymo.
But Tesla’s self-driving technology has remained under the scrutiny of regulators. FSD relies on multiple onboard cameras to feed machine-learning models that, in turn, help the car make decisions based on what it sees.
Meanwhile, Waymo’s technology relies on premapped roads, sensors, cameras, radar, and lidar (a laser-light radar), which might be very costly, but has met the approval of safety regulators.

Read more
Are self-driving cars the death of car ownership?
Tesla Cybercab at night

Self-driving cars are coming. It remains to be seen how long that will take. Plenty of vehicles can more or less drive themselves on highways, but for now, they still can't completely reliably drive themselves on all streets, in all conditions, taking into account all different variables. One thing is clear, though: the tech industry sees autonomous driving as the future of personal transportation, and they're spending billions to reach that goal.

But what happens when we get there? Tesla made headlines for not only announcing its new Cybercab fully autonomous vehicle, but simultaneously claiming that customers will be able to buy one. That's right, at least if Tesla is to be believed, the Cybercab doesn't necessarily represent Tesla building its own Uber-killing fleet of self-driving cars, but instead giving people the ownership over the self-driving car industry.

Read more
Tesla Autopilot vs. full self-driving: What’s the difference?
A Telsa Model 3 drives along a road.

It's no longer the only company with self-driving cars on the road, but Tesla was one of the first brands to make this innovative functionality available to the public. Thanks to an array of cameras, sensors, and AI technology, most Telsa vehicles are capable of driving themselves to some degree. However, this doesn't mean drivers can take a nap behind the wheel. In fact, none can be used without driver supervision -- and there are some serious limitations to the tech.

Tesla currently offers features known as Tesla Autopilot and Full Self-Driving. But what's the difference between the two? And is one more reliable than the other? Here's everything you need to know about Tesla's Autopilot and Full Self-Driving technology.
Tesla Autopilot

Read more