Skip to main content

In Google’s new world, your phone camera is for way more than taking photos

If you’re like most people, you probably use your phone’s camera to snap vacation photos, brag about your latest kitchen creations, and obsess over your pets and kids. How cute. If Google has its way, you might as well be using a $3,000 commercial gas range to boil water for mac and cheese.

At Google I/O 2021, the company’s annual developer conference, the software behemoth showed off a range of new apps and tools that treat your smartphone camera less as a humble memory maker, and more as a supercharged tool for interacting with the world around you.

Google Lens allows you to shop and make purchases merely by pointing a camera at your surroundings.
Image used with permission by copyright holder

Take three quick shots of those weird red bumps that just emerged on your arm, for instance, and Google has a new tool that will search for matching dermatological conditions. It uses machine learning and a few cues from you (how old are you, how long have you had it?) to identify the likely culprit in seconds.

The steadily improving object recognition in Google Lens has allowed Google to recognize what’s in a photo for years, but advances in machine learning now allow you to do much more than simply ask, “What is this?” Taking a photo of your hiking boots and asking Google “Can I hike Mt. Fuji in these?” is now a quick way to answer a search query that doesn’t exactly lend itself to text. And naturally, Google would be happy to help you buy just about anything you can point a camera at. 

Your camera can do everything from help you shop to help you diagnose a skin condition.

Augmented reality is also moving beyond its cutesy phase of adding hats to video chats, and transforming into a must-have tool. Hold your phone up when you’re on the hunt for a restaurant, for instance, and Google Maps Live View will superimpose directions on top of your camera’s viewfinder. Google introduced this feature to Maps in 2019, but expanded it to more than 100 countries this year, turning it from a proof of concept to a legitimate navigational aid. And now it will work indoors, for when you need to find your airport gate or an urgently needed bubble tea at the mall.

Dermatology diagnosis from Google
Google

In Google’s camera-centric new world, you can even use a photo to transform the entire look of your phone’s operating system. A new feature in Android 12 will analyze the colors in your lock-screen image to generate a custom color palette for everything from backgrounds to buttons in your apps.

Not all of these features are new, but the widening scope of what is possible shows Google’s clear trajectory: A world where your camera is every bit as much an input device as the dusty mouse and keyboard next to your desktop. Just as voice assistants have made it second nature to ask Alexa aloud for calculations, cooking tips, and alarms, Google wants you to reach for your phone camera not just when you want to capture a memory, but when you have a question.

Privacy advocates may wince at giving Google yet another pipeline of precious data to poke and prod, slice and dice. And Google even admitted that some of these shopping technologies aim to “turn the world into your personal showroom.” But the next time your flight leaves in seven minutes and you can’t find your gate, you might not care as you whip open Google Maps to get there.

Editors' Recommendations

Nick Mokey
As Digital Trends’ Managing Editor, Nick Mokey oversees an editorial team delivering definitive reviews, enlightening…
The 10 best apps for your 2024 New Year’s resolutions
An app folder on an iPhone titled "New Year's Apps."

Now that the holidays have come and gone, it’s time to look forward to the new year. And you know what that means — 2024 New Year’s resolutions!

I know that we all try to make some resolutions each year, but it’s hard to stay on top of it. Whether it’s trying to eat healthier and exercise more, managing your budget better, or even just trying to form better overall habits and break bad ones, there are apps to help you stay on track.

Read more
Your Android phone is getting lots of fun new features this month
Android 14 logo on the Google Pixel 8 Pro.

It’s a big day for Android updates, as Google has announced more than a dozen new features spread across multiple devices and apps — including a host of fun new ways to make Google Messages more visually exciting and to mark a significant milestone in its use.

Ready to see everything that's new? Let's dig in.
Google Messages

Read more
Google just redesigned one of its biggest apps, and it’s bad
Google Chat app on the Play Store.

Google Chat — Google's business-oriented messaging platform that is similar to Slack and Microsoft Teams — just got a big update for its Android and iOS apps. The update dramatically changes how you navigate the app and, uh, well, it sure is something.

Google Chat's mobile app used to be broken up into two pages: Chat (direct messages between you and other users) and Spaces (larger chat rooms for multiple people). As with most apps, you switched between these with a navigation bar at the bottom of your screen.

Read more