Skip to main content

How crowdsourced lidar could give your car X-ray-like superpowers

One of my uncles always tells a story about how, when he was a kid, his mom would tell him she could see around corners. They would be out walking someplace, only for his mom to tell him the details of some vehicle or person that was about to appear around a bend in the road. A few seconds later and, sure enough, that vehicle or person would appear, exactly as described. Magic, surely?

Of course, it wasn’t magic at all: His mom — my grandmother — was just taller than he was, and could see over walls and other obstacles that he wasn’t able to. What appeared to be some kind of superpower was really just about having a superior vantage point.

Now, researchers from the U.K.’s University of Cambridge, University of Oxford, and University College London want to give every car on the planet the ability to see around corners. And, with genuine magic being in short supply, they’ve come up with a way to shift the world’s vantage points using a combination of lidar, augmented reality, and crowdsourcing.

If it works as promised — and that’s a big if — it could totally transform the way we drive by allowing drivers to “see through” objects to alert them of potential hazards, without distracting them in the process. And it will also “beam” the information directly into your eye for good measure.

Crowdsourcing lidar

Lidar (Light Detection And Ranging) refers to the depth-sensing, bounced laser-mapping technology that allows many self-driving cars to perceive the world around them. As it happens, those last four words — “the world around them” — is the bit that the researchers behind this project want to change. To give drivers something akin to X-ray vision that allow them to spot obstacles hidden from view — such as the motorcyclist momentarily obscured behind a vehicle — they want to build a massive crowdsourced map of lidar data gathered from all road users.

University of Cambridge

For an analogy of what this might look like, think of that scene from Christopher Nolan’s 2008 movie The Dark Knight in which Batman hacks every cell phone in Gotham City and converts them into a high-frequency generator, stitching together all the location data to build a three-dimensional schematic of the city, from buildings to people. As Lucius Fox, the perturbed Wayne Enterprises boss, says, “you took my sonar concept and applied it to every phone in the city. Half the city feeding you SONAR; you can image all of Gotham.”

The idea of car-to-car communication for collaborative purposes isn’t exactly science fiction. Starting with Waze, many mapping apps have used the driving data of different users to build up a pretty detailed picture of what is happening on the road in terms of the free flow of traffic. Tesla, meanwhile, collects large amounts of road data from vehicle owners via its Full Self-Driving beta test fleet. In 2017, Tesla asked vehicle owners if they were willing to provide videos collected using their cars’ onboard Autopilot cameras. This data, while collected by individual vehicles, is combined to make the overall fleet smarter and better able to deal with obstacles.

What this latest LiDAR project adds to that is the gathering of 360-degree point cloud data that can be aggregated to give every road user a clear view of their surroundings.

As Jana Skirnewskaja, a researcher on the team, told Digital Trends, this is still relatively early times for the project. So far, the team has carried out a proof of concept scanning Malet Street, a busy street in London, using multiple lidar scanners in various positions. This data was then used to build up a 3D model.

3D Model of Malet St, Central London, Based on LiDAR Data

“We scanned Malet Street from 10 different positions using 10 different data scanners,” Skirnewskaja told Digital Trends. “This allows us to fully re-create the street how it is at that moment, so any objects — hidden or not — will be [represented in] the point cloud. This allows us to erase objects that we don’t want to see, and choose the objects which are hidden … and project them.”

Beaming the information into drivers’ eyes

As it happens, this is only one half of the project. The other, equally impressive, bit involves projecting this information directly into the eye of the driver in ultra-high definition. This in-car technology, Skirnewskaja believes, could be a valuable alternative to 2D windscreen AR projection, as well as to burgeoning AR tech like augmented reality contact lenses.

“What our studies have shown is that it [causes] no harm at all to the pupil, to the human eye,” she said. “It can project, [directly] into the driver’s eye, any object. We can also use augmented reality to layer objects so that we project different objects, like road obstacles or signs or people or trees, at different sizes [to indicate] distances. The further away an object is, the smaller it will be. That can be realized.”

University of Cambridge

It means that, as a driver sits behind the wheel, they could have overlaid information appear superimposed on the real world. “[Our work] has shown that we can already project in-eye 3D augmented reality objects on the road, and that these are properly aligned and not distracting the driver,” Skirnewskaja said.

She said that, initially, this will likely be fixed information, such as highlighting permanent obstacles that have caused other drivers problems. But, in the long term, it could be possible to track dynamic objects as well. In addition to gathering lidar data from other vehicles, Skirnewskaja said that cities could install lidar sensors along the sides of roads, similar to the way CCTV cameras are used today.

“We hope that it can be expanded further so that we can connect every car and project this information of road obstacles in real time,” she explained.

The team aims to work with established automotive companies as part of the project. She suggested that this includes Jaguar Land Rover and VW. At present, the researchers are working to miniaturize the optical components they used in their experimental holographic setup so that it can be fitted into a car. After this, they plan to carry out vehicle tests on public roads in the city of Cambridge.

There’s no word on when this technology ultimately goes live, but, provided it works as well as described, it’ll certainly be worth waiting for.

A paper describing the work was recently published in the journal Optics Express.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
EV vs. PHEV vs. hybrid: What’s the difference?
BMW X5 PHEV charge port

When sizing up options for your next car, you may be figuring out whether to get an electric vehicle, only to discover there are a bunch of variations to consider -- not just hybrids, but plug-in hybrids, extended-range electric vehicles, and fuel cell electric vehicles are just some of the other categories. The depths of EV jargon run so deep that we wrote an entire EV glossary, but for now let's zero in on the difference between electric vehicles, hybrids, and plug-in hybrids. These options blend old tech and new tech in a way that's often practical, cheaper than an EV, and still more efficient than an old-school gasoline car.
What is an electric vehicle?
An electric vehicle skips the internal combustion engine found in most traditional cars in favor of an electric motor. This allows EVs to operate without needing gasoline. Instead, they're powered by an electric battery that will need to be charged regularly, either at your home or at a charging station like a Tesla Supercharger. The Ford Mach-E, Kia EV6, and Rivian R1S are all popular examples of modern EVs.

The electric motor works by way of a rotating magnetic field. Inside the motor, three electromagnets surround a free-floating rotor, which spins based on which magnet is attracting it most. That rotor in turn produces power to the wheels of the car and pushes it forward and backward. Regenerative braking reverses the relationship and turns motion into electricity. While you're slowing to a stop, the force of the turning wheels spins the rotor and generates a charge via the electromagnets in the motor, which in turn goes up into the battery for storage. If you're curious, you can dig into the nuts and bolts of how an electric vehicle works.
What's the difference between a hybrid and a plug-in hybrid?
In short, a hybrid primarily relies on gas with an electric backup, while a plug-in hybrid relies on electric power with a gas backup.

Read more
You’ll soon be able to watch YouTube videos in your Android Automotive car
Android Auto in a car.

Google is making a bigger play for the in-car infotainment system. At Google I/O 2023, the company took the wraps off of a series of improvements to both Android Auto and Android Automotive, allowing those who want Google-based services in their car to get more features and better account integration.

As a reminder, the two systems may have a similar (almost identical?) name, but are actually quite different. Android Auto essentially just projects content from your phone, whether through a wireless or wired connection. It's Google's answer to Apple's CarPlay, and doesn't work without your phone. Android Automotive, however, is a version of Android that runs in the car itself, as the car's main infotainment system. It works whether you have a connected phone or not. Collectively, Google refers to the systems as Android for Cars -- yes, yet another name.

Read more
Are EVs safe? From battery fires to autopilot, here are the facts
Lucid Air electric car

While many people will be primarily concerned with EV range before buying their first electric vehicle, others are a little nervous about having a giant lithium-ion battery strapped to their car's undercarriage. Those things can catch fire -- just ask Chevy Bolt owners. But how much of a real danger is that? And should it prevent you from buying an EV?
What safety features do EV batteries have?
The major safety issue with lithium-ion batteries is their temperature. If they get too hot, they're prone to igniting. If they get too cold, they freeze and permanently stop working. Charge and discharge rates need to be carefully regulated too, or you'll get electrical fires. Over time, small imperfections in a battery's structure can lead to short circuits and reduced lifetime.

EVs have what are called battery management systems (BMS) to keep tabs on all of these variables. The BMS will generate warnings when needed and intervene directly by cutting off power if things get out of hand. EV battery packs also have thermal management systems. Typically, this is a closed loop of liquid coolant flowing alongside the battery cells, but air cooling and welding battery cells directly to the car chassis are also means of mitigating extreme heat.
How well do EVs handle a crash?
Since there's no engine at the front of an EV, the hood typically houses a frunk -- meaning a front trunk. This acts as a large crumple zone in the case of a head-on accident. One crash in Germany avoided casualties thanks to this inherent characteristic of electric vehicles. Crash tests bear this out. Popular EVs like the Tesla Model 3, Hyundai Ioniq 5, and Nissan Leaf have all received overall five-star ratings from the National Highway Traffic Safety Administration (NHTSA).

Read more