The newest phones use a time-of-flight camera, but what is it? We ask an expert

LG G8 ThinQ
Julian Chokkattu/Digital Trends

The race to improve and differentiate smartphones has been focusing more and more on the camera suite and all that it offers. There’s a good chance that you’ve heard the recent buzz surrounding time-of-flight cameras, as manufacturers are starting to put them into phones. But what exactly is it?

Time of Flight

Time-of-flight (ToF) cameras are comprised of a sensor that uses a tiny laser to fire out infrared light. This light bounces off anything or anyone in front of the camera and back into the sensor. The length of time the light takes to bounce back is measured and that translates to distance information that can be used to create a depth map.

“Time-of-flight cameras are actually measuring the time it takes for light to go from the camera to the user or environment and be reflected back to the sensor,” Dr. Xavier Lafosse, commercial technology director of Corning Precision Glass Solutions, told Digital Trends.

Corning

Up until now, most phones have relied on stereovision, which employed two cameras to calculate rough depth, but this method doesn’t work in low light or in the dark and it’s not very accurate.

A better method that also employs infrared is structured light illumination where a dot pattern is projected onto a scene or face and the sensor measures the distance between the dots and looks at the distortion in the pattern to calculate depth. This technology works well in the short range, up to arm’s length, for things like facial recognition, which is why Apple employed it with its TrueDepth Camera for Face ID.

Time of flight works in a similar way, but it doesn’t use a pattern of dots. Because these methods rely on infrared light, they work well in low light and even dark environments. The time-of-flight camera illuminates the scene with a homogenous flood of light and the camera looks at every individual pixel in the image. The sensor synchronizes with an incredibly sensitive clock that’s capable of measuring tiny variations revealed by the speed of light bouncing back. With depth information assigned to every pixel you get a rich depth map.

“It’s the only method that’s really accurate at measuring distance.”

“It’s the only method that’s really accurate at measuring distance,” Dr. Lafosse said. “It’s the only one that’s not an interpolation or a calculation, but rather a measure of distance.”

There are various different potential applications for accurate depth mapping like this, which is why it’s creeping into more phones. You’ll find a time-of-flight camera in the LG G8 ThinQ and in the Honor View 20, to give two recent examples, but the implementation is different.

LG has paired the time-of-flight sensor with its 8-megapixel front-facing camera to create what it’s calling the Z Camera — the Z-axis denotes depth for 3D images. This enables facial unlock and something called Hand ID, another secure biometric, which reads the vein patterns in your hand. It’s also used for Air Motion gestures, allowing you to wave your hand over your G8 ThinQ to trigger various actions such as playing and pausing music without touching the device.

LG G8 ThinQ
Julian Chokkattu/Digital Trends

In the Honor View 20 the time-of-flight camera is paired with a 48-megapixel sensor as part of the main, rear-facing camera. It lends depth information that enhances portrait mode, creating a really accurate bokeh effect with the subject in sharp relief and the background blurred. But that’s not all it can do as part of a phone’s main camera.

“The value of time of flight is really about the midrange to long range distances; think about applications like augmented reality,” Dr. Lafosse said. “If the time-of-flight camera is on the back, then you know it’s not about facial recognition, but about sensing the environment and what you have in front of you.”

You may have tried out some augmented reality apps and games in the past, but time-of-flight cameras can dramatically boost the accuracy and fuse your actual environment with gameplay and characters for a whole new level of experience. That may be shooting zombies in your hallway or seeing how a piece of furniture you’re thinking about buying would really look in your living room.

There’s potential for deeper social interactions as well. Instead of FaceTime with Animojis, you might have a more fully realized 3D experience.

“Your friend’s avatar isn’t floating in the air, but actually sitting on your couch next to you.”

“Time-of-flight sensors can map your environment accurately, so your friend’s avatar isn’t floating in the air, but actually sitting on your couch next to you,” Dr. Lafosse said.

The underlying technology isn’t new. There was a time-of-flight camera in Microsoft’s Kinect sensor and the military has been using time-of-flight technology to get depth information for many, many years. But Dr. Lafosse said that improvements in the technology have allowed integration of the required elements into ever smaller form factors, and new applications for it are driving its adoption in phones.

This technology is also vital for augmented reality or mixed reality wearables — like Microsoft’s HoloLens or Magic Leap — to work because these systems need a very accurate picture of your environment.

hololens 2 news roundup dynamics 365 guides holographic training 4 1920x1080
Microsoft

Another area where time-of-flight cameras could help is indoor navigation. If there’s a 3D map of your building in the cloud, then the sensor could potentially recognize precisely where you are at any given moment.

Why does Corning know all this? The company makes the glass that protects most smartphones, and it’s also working on the optical components of time-of-flight cameras — making them smaller, more transparent, and ensuring they perform as well as possible — while manufacturers like Sony continue to improve the sensors, making them smaller and more power efficient. We are sure to see time-of-flight camera appearing in more and more smartphones in the near future, and there are strong rumors that Apple will include one in the next iPhone.

Augmented reality was exciting when it first emerged, but upon trying it out it was hard to escape the feeling that the technology failed to live up to the hype. Time-of-flight cameras, coupled with improved processing power and higher speed connectivity, could be about to bring that original vision to life.

Product Review

It's a shame the U.S. banned Huawei. The new Honor 20 Pro is a kick-ass phone

Where does Honor go after the Honor View 20, the best device it has ever made? The answer is the Honor 20 Pro, which takes what made the View 20 great, and then improves on it by adding more camera lenses and shrinking the size.
Photography

What are mirrorless cameras and are they better than DSLR cameras?

What exactly is a mirrorless camera, and what makes them so special? In this article, we break down the specifics of mirrorless cameras, including how they differ from the likes of a DSLR camera, and what kind of pros and cons are.
Product Review

Oppo's RX17 Pro is a good smartphone up against great competitors

The Oppo RX17 Pro looks great, and has a triple-lens camera on the back; but don’t let these aspects make you rush out and buy one, especially with the masses of excellent devices that are vying for your attention.
Android

Get ready for 5G speed: The Samsung Galaxy S10 5G has landed in the U.S.

Samsung announced a whopping four new Galaxy S10 devices, from the low-cost S10e to the triple-camera S10 and S10 Plus. But it's the Galaxy S10 5G that steals the show, as it's the first 5G-ready smartphone to hit the market.
Mobile

Cardinal red Galaxy S10 doesn't appear destined for the U.S.

Samsung 2019 flagship smartphone lineup is here, and there aren't just two phones as usual — there are four. There's the Galaxy S10, S10 Plus, as well as a new entry called the S10e, as well as the Galaxy S10 5G.
Mobile

Sony not giving up on smartphones, but will only focus on 4 regions

Sony says it will not give up on phones, and still considers them indispensable to its business, but will focus on only four regions around the world — Japan, Europe, Taiwan, and Hong Kong.
Deals

REI cuts prices on Garmin Fenix 5 and Suunto 3 fitness smartwatches

REI is currently dropping prices on Garmin Fenix and Suunto smartwatches for its anniversary sale. With fitness tracking, GPS, and smartphone connectivity, a solid wearable is one of the best companions outdoorsmen could wish for.
Deals

Best Memorial Day sales 2019: The top deals from Walmart, Dell, and Home Depot

If you're looking to save big on some shiny new stuff for Memorial Day 2019, we've gathered everything you need to know into one place. Find out where to save the most money before the summer hits its stride.
Mobile

Chip architecture firm ARM ceases business with Huawei, claims latest leak

The U.S. Commerce Department has added Huawei to its "Entity List." Google, Intel, and ARM are all confirmed or rumored to be ceasing business with the company, which may have disastrous effects on Huawei.
Mobile

Keep your gadgets fully powered with the best Micro USB cables

A lot of older gadgets and Android smartphones still rely on Micro USB cables to charge or transfer data. Check out our picks of the best Micro USB cables, whether you want something reliable, tough, feature-packed, or simple.
Android

Keep up-to-date with the best news apps on iPhone and Android

Are the days of traditional newspapers and broadcast news dwindling? With apps this good, maybe. Catch up on the latest headlines on any platform with the best news apps on iOS and Android.
Mobile

Looking for love or just some fun? Cozy up with the best dating apps of 2019

Everyone knows online dating can be stressful, time-consuming, and downright awful. Check out our top picks for the best dating apps, so you can streamline the process and find the right date, whatever you're looking for.
Mobile

Embattled Huawei preps its own backup operating system that runs Android apps

According to a new report, Huawei is developing its own mobile operating system, just in case it loses its access to Android -- something that could happen to ZTE in the near future.
Computing

Apple sends out invites for WWDC 2019, and unicorns are involved

Apple developers and fans alike look forward every year to the company's Worldwide Developers Conference, better known as WWDC. Apple has confirmed the conference will take place on June 3-7, and the company just sent out invites.