Google Lens adds unprecedented intelligence to your smartphone camera

how to use google lens keynote 4
Google Lens computer vision technology lets you find context with what your camera sees.

Want to know the name of that flower or bird you encounter during your stroll through a park? Soon, Google Assistant will be able to tell you, using the camera and artificial intelligence.

Google jump-started its 2017 I/O conference around AI and machine learning, and one computer vision technology it highlighted is Google Lens, which lets the camera do more than just capture an image — it gives greater context around what it is that you’re seeing.

Coming to Google Assistant and Google Photos, the Google Lens technology can “understand what you’re looking at and help you take action,” Google CEO Sundar Pichai said during the keynote. For example, if you point the camera at a concert venue marquee, Google Assistant can tell you more about the performer, as well as play music, help you buy tickets to the show, and add it to your calendar, all within a single app.

When the camera’s pointed at an unfamiliar object, Google Assistant, through image recognition, can tell you what it is. Point it at a shop sign, and using location info, can give you meaningful information about the business. All this can be done through the “conversational” voice interaction the user has with Assistant.

“You can point your phone at it and we can automatically do the hard work for you,” Pichai said.

If you use Google’s Translate app, you have already seen how the technology works: Place a camera over some text and the app will translate it to a language you understand. In Google Assistant, Google Lens will take this further. In a demonstration, Google showed that Google Assistant not only will translate foreign text, but also display images of what the text is describing, to give more information.

google lens io 2017 keynote 1
In a demo, Scott Huffman, Google VP of engineering for Assistant, demoed how Google Lens within Google Assistant can translate the Japanese text of an image, but also give further context about what the word is.

Image recognition technology isn’t new, but Google Lens shows how advanced machine learning is becoming. Pichai said that as with its work on speech, Google is seeing great improvements in vision. The computer vision technology not only helps recognize what something is, but can even help repair or enhance an image. Took a blurry photo of the Eiffel Tower? Because the computer recognizes the object and knows what it’s suppose to look like, it can automatically enhance that image based on what is already knows.

“We can understand the attributes behind a photo,” Pichai said. “Our computer vision systems now are even better than humans at image recognition.”

google lens io 2017 keynote 2
No longer will you need to write down what’s in your vacation photos. Google VP Anil Sabharwal for Google Photos showed how Google Lens can recognize objects in a photo and bring up relevant information about it.

To make Lens effective at its job, Google is employing sophisticated computational architecture of Cloud Tensor Processing Unit (TPU) chipsets, to handle training and inference for its machine learning. Its second-generation TPU technology can handle 180 trillion floating point operations per second; 64 TPU boards in one super computer can handle 11.5 petaflops. With this computing power, new TPU can handle both training and inference simultaneously, which wasn’t possible in the past (the previous TPU could only handle inference work, but not the more complex training). Machine learning takes time, but this hardware will help accelerate the effort.

Google Lens will also power the next update of Google Photos. Image recognition is already used in Photos to recognize faces, places, and things to help with organization and search. With Google Lens, Google Photos can give you greater information about the things in your photos, like the name and description of a building; tapping on a phone number in a photo will place a call, pulling up more info on an artwork you saw in a museum, or even enter the Wi-Fi password automatically from a photo you took of the back of a Wi-Fi router.

google lens io 2017 keynote 3
Hate entering Wi-Fi network passwords? Snap a photo of the wireless settings, and Google Lens technology through Google Photos can automatically enter it for you.

Assistant and Photos will be the first apps to use Google Lens, but it will be rolled out into other apps. And with the announcement of support for Assistant in iOS, iPhone users will be able to utilize the Google Lens technology as well.

Deals

Walmart drops prices on Google Home Hub and Lenovo smart displays

Walmart slashed the prices of Google Home smart displays for two highly regarded Lenovo models and the original Google Home Hub. We can't think of a better way to improve your smart home setup and these deals can help you save up to $100.
Mobile

Google Fi: Phones, plans, pricing, perks, and more explained

Google's wireless service, formerly Project Fi, now goes by the name of Google Fi, and it's now compatible with a majority of Android phones, as well as iPhones. Here's everything you need to know about Google Fi.
Gaming

Here’s all the news from the Google Stadia GDC presentation

Google announced its new game streaming service Stadia during GDC 2019. Here is all the news from the event, including news on YouTube streaming and quickly accessing your favorite new games.
Gaming

Is Google launching a game-streaming service? Watch our GDC livestream to see

Google is presenting its 2019 GDC keynote on March 19. Here is how you can tune in to watch the event, as well as what could be shown during the presentation. Google's Project Stream service could be on the agenda.
Social Media

A Facebook, Instagram bug exposed millions of passwords to its employees

Facebook, Facebook Lite, and Instagram passwords weren't properly encrypted and could be viewed by employees, the company said Thursday. The network estimates millions of users were affected.
Mobile

Diesel’s denim-inspired smartwatch straps are a casual, colorful must-own

Diesel will release two new versions of the On Full Guard 2.5 smartwatch later this year, with seriously cool, denim-inspired straps in classic Diesel colors. We tried them on at the Baselworld 2019 show.
Movies & TV

Apple’s next big event is set for March 25: Here’s what you can expect

Apple's next big event takes place on March 25 in Cupertino, California. The company is expected to make several announcements related to its services, including Apple TV, so follow our guide to get ready for the big event.
Wearables

Tips and tricks to get you started with your new Fitbit Inspire HR

The Inspire HR may be an entry-level fitness tracker in Fitbit's lineup, but the device still has plenty of features to explore. These are our favorite tips and tricks to help you use the Inspire HR to its fullest potential.
Mobile

How three simple words could be the difference between life and death

What3Words’ app-based address system gives a three-word code to every three-meter-square patch on the planet, with its accuracy and ease of use now catching the attention of first responders in the U.K.
Mobile

The Moto G7 Power, with its massive battery, is now available for purchase

After a number of leaks and rumors, the Motorola Moto G7, Moto G7 Play, and Moto G7 Power are finally here. The devices represent quite a spec bump over the previous-generation Moto G6 phones, yet still come at a reasonable price.
Mobile

Got gadgets galore? Keep them charged up with the 10 best USB-C cables

We're glad to see that USB-C is quickly becoming the norm. That's why we've rounded up some of the better USB-C cables on the market, whether you're looking to charge or sync your smartphone. We've got USB-C to USB-C and USB-C to USB-A.
Computing

Get ready to say goodbye to some IFTTT support in Gmail by March 31

If This Then That, the popular automation service, will drop some of its support for Gmail by March 31. The decision comes as a response to security concerns and is aimed to protect user data.
Mobile

5G's arrival is transforming tech. Here's everything you need to know to keep up

It has been years in the making, but 5G is finally becoming a reality. While 5G coverage is still extremely limited, expect to see it expand in 2019. Not sure what 5G even is? Here's everything you need to know.
Mobile

This is the easiest way to save your iPhone data to your computer

Living in fear of losing your contacts, photos, messages, and notes on your iPhone? Fear no more -- in this guide, we'll break down exactly how to back up your iPhone to your computer using Apple's iTunes or to the cloud with iCloud.