The newest phones use a time-of-flight camera, but what is it? We ask an expert

LG G8 ThinQ
Julian Chokkattu/Digital Trends

The race to improve and differentiate smartphones has been focusing more and more on the camera suite and all that it offers. There’s a good chance that you’ve heard the recent buzz surrounding time-of-flight cameras, as manufacturers are starting to put them into phones. But what exactly is it?

Time of Flight

Time-of-flight (ToF) cameras are comprised of a sensor that uses a tiny laser to fire out infrared light. This light bounces off anything or anyone in front of the camera and back into the sensor. The length of time the light takes to bounce back is measured and that translates to distance information that can be used to create a depth map.

“Time-of-flight cameras are actually measuring the time it takes for light to go from the camera to the user or environment and be reflected back to the sensor,” Dr. Xavier Lafosse, commercial technology director of Corning Precision Glass Solutions, told Digital Trends.

Corning

Up until now, most phones have relied on stereovision, which employed two cameras to calculate rough depth, but this method doesn’t work in low light or in the dark and it’s not very accurate.

A better method that also employs infrared is structured light illumination where a dot pattern is projected onto a scene or face and the sensor measures the distance between the dots and looks at the distortion in the pattern to calculate depth. This technology works well in the short range, up to arm’s length, for things like facial recognition, which is why Apple employed it with its TrueDepth Camera for Face ID.

Time of flight works in a similar way, but it doesn’t use a pattern of dots. Because these methods rely on infrared light, they work well in low light and even dark environments. The time-of-flight camera illuminates the scene with a homogenous flood of light and the camera looks at every individual pixel in the image. The sensor synchronizes with an incredibly sensitive clock that’s capable of measuring tiny variations revealed by the speed of light bouncing back. With depth information assigned to every pixel you get a rich depth map.

“It’s the only method that’s really accurate at measuring distance.”

“It’s the only method that’s really accurate at measuring distance,” Dr. Lafosse said. “It’s the only one that’s not an interpolation or a calculation, but rather a measure of distance.”

There are various different potential applications for accurate depth mapping like this, which is why it’s creeping into more phones. You’ll find a time-of-flight camera in the LG G8 ThinQ and in the Honor View 20, to give two recent examples, but the implementation is different.

LG has paired the time-of-flight sensor with its 8-megapixel front-facing camera to create what it’s calling the Z Camera — the Z-axis denotes depth for 3D images. This enables facial unlock and something called Hand ID, another secure biometric, which reads the vein patterns in your hand. It’s also used for Air Motion gestures, allowing you to wave your hand over your G8 ThinQ to trigger various actions such as playing and pausing music without touching the device.

LG G8 ThinQ
Julian Chokkattu/Digital Trends

In the Honor View 20 the time-of-flight camera is paired with a 48-megapixel sensor as part of the main, rear-facing camera. It lends depth information that enhances portrait mode, creating a really accurate bokeh effect with the subject in sharp relief and the background blurred. But that’s not all it can do as part of a phone’s main camera.

“The value of time of flight is really about the midrange to long range distances; think about applications like augmented reality,” Dr. Lafosse said. “If the time-of-flight camera is on the back, then you know it’s not about facial recognition, but about sensing the environment and what you have in front of you.”

You may have tried out some augmented reality apps and games in the past, but time-of-flight cameras can dramatically boost the accuracy and fuse your actual environment with gameplay and characters for a whole new level of experience. That may be shooting zombies in your hallway or seeing how a piece of furniture you’re thinking about buying would really look in your living room.

There’s potential for deeper social interactions as well. Instead of FaceTime with Animojis, you might have a more fully realized 3D experience.

“Your friend’s avatar isn’t floating in the air, but actually sitting on your couch next to you.”

“Time-of-flight sensors can map your environment accurately, so your friend’s avatar isn’t floating in the air, but actually sitting on your couch next to you,” Dr. Lafosse said.

The underlying technology isn’t new. There was a time-of-flight camera in Microsoft’s Kinect sensor and the military has been using time-of-flight technology to get depth information for many, many years. But Dr. Lafosse said that improvements in the technology have allowed integration of the required elements into ever smaller form factors, and new applications for it are driving its adoption in phones.

This technology is also vital for augmented reality or mixed reality wearables — like Microsoft’s HoloLens or Magic Leap — to work because these systems need a very accurate picture of your environment.

hololens 2 news roundup dynamics 365 guides holographic training 4 1920x1080
Microsoft

Another area where time-of-flight cameras could help is indoor navigation. If there’s a 3D map of your building in the cloud, then the sensor could potentially recognize precisely where you are at any given moment.

Why does Corning know all this? The company makes the glass that protects most smartphones, and it’s also working on the optical components of time-of-flight cameras — making them smaller, more transparent, and ensuring they perform as well as possible — while manufacturers like Sony continue to improve the sensors, making them smaller and more power efficient. We are sure to see time-of-flight camera appearing in more and more smartphones in the near future, and there are strong rumors that Apple will include one in the next iPhone.

Augmented reality was exciting when it first emerged, but upon trying it out it was hard to escape the feeling that the technology failed to live up to the hype. Time-of-flight cameras, coupled with improved processing power and higher speed connectivity, could be about to bring that original vision to life.

Wearables

The Tacs Nato-Lens is a watch for those who live life through a camera lens

Not all watches have to be round, or connect to your phone. The Tacs Nato-Lens is a stylish piece of wristwear with a design inspired by classic SLR cameras, making it a geeky timepiece for the camera and watch fan.
Smart Home

These best outdoor security cameras will keep porch pirates at bay

Worried about porch pirates stealing your packages, or intruders entering your home? Always be in the know about who or what is on your property by installing one of these outdoor security cameras.
Product Review

What do you do with 187 megapixels? The Lumix S1R is glorious overkill

The Lumix S1R is one of the most capable cameras ever made, from its robust build to extensive feature set. But its key feature, a 187MP high resolution mode, is something few customers will have use for.
Mobile

The Samsung Galaxy S10 5G might be a few short weeks away from launch

Samsung has announced a whopping four new Galaxy S10 devices, from the low-cost S10e to the triple-camera S10 and S10 Plus. But it's the Galaxy S10 5G that steals the show as it's among the first 5G-ready smartphones to hit the market.
Mobile

The Moto G7 Power, with its massive battery, is now available for purchase

After a number of leaks and rumors, the Motorola Moto G7, Moto G7 Play, and Moto G7 Power are finally here. The devices represent quite a spec bump over the previous-generation Moto G6 phones, yet still come at a reasonable price.
Mobile

Got gadgets galore? Keep them charged up with the 10 best USB-C cables

We're glad to see that USB-C is quickly becoming the norm. That's why we've rounded up some of the better USB-C cables on the market, whether you're looking to charge or sync your smartphone. We've got USB-C to USB-C and USB-C to USB-A.
Computing

Get ready to say goodbye to some IFTTT support in Gmail by March 31

If This Then That, the popular automation service, will drop some of its support for Gmail by March 31. The decision comes as a response to security concerns and is aimed to protect user data.
Mobile

5G's arrival is transforming tech. Here's everything you need to know to keep up

It has been years in the making, but 5G is finally becoming a reality. While 5G coverage is still extremely limited, expect to see it expand in 2019. Not sure what 5G even is? Here's everything you need to know.
Mobile

This is the easiest way to save your iPhone data to your computer

Living in fear of losing your contacts, photos, messages, and notes on your iPhone? Fear no more -- in this guide, we'll break down exactly how to back up your iPhone to your computer using Apple's iTunes or to the cloud with iCloud.
Mobile

HMD Global admits Nokia 7 Plus handsets sent user data to China

Nokia could be in some hot water. According to recent reports, Nokia 7 models may be secretly sending data to China without the user knowing about it. Nokia says that the issue was a software bug and that it has been fixed.
Mobile

Here are the best iPad Pro keyboard cases to pick up with your new tablet

The iPad Pro range can double as laptops, but they do need proper keyboards to fill in effectively. Thankfully, there are loads to choose from and we rounded up the best iPad Pro keyboard cases right here.
Mobile

24 must-have apps for rooted Android phones and tablets

Rooting your Android device opens up a world of possibilities, along with a few apps. Here are 24 of our favorites, so you can make the most of your rooted device and unleash the true power of Android.
Mobile

Wring the most out of iOS with the best Siri commands

You may not know all the things you can say to Siri -- after all, Apple never released an official list of commands for its virtual assistant. Thankfully, we've compiled a list of the best Siri commands to help you out.
Product Review

The new iPad Mini isn’t a beauty, but it performs like a beast

Apple’s new iPad Mini has beastly performance, fluid iOS 12 software, and good battery life. It also looks like it came straight out of 2015, because the design hasn’t been changed. Here's our review of Apple’s 7.9-inch tablet.