Android’s Now on Tap feature, Google’s melding of image recognition and machine learning in an app, was, at least at first, a lot more impressive in theory than in practice. Sure, you could use it to pull up showtimes for that movie you saw mentioned on Twitter or summon a list of song titles for that Queen album your friend texted, but that was the extent of it. In the intervening months, though, Now on Tap has become much more capable — and useful.
In December, it gained the ability to track package and flight info. In June, it picked up support for image searches straight from your phone’s camera — you can take a picture of a painting or monument and Google will do its darnedest to identify it. And on Wednesday, Now on Tap received one of its biggest updates to date: instant text translation, a new Discover section, and support for QR code and bar codes.
Text translation is perhaps the most welcome addition. After Wednesday’s update, Now on Tap will have the linguistic chops to recognize multiple languages within Chrome, Hangouts, Snapchat, or any other app you might find yourself using, Google Search product manager Aneto Okonkwo said in a blog post. It works a bit like Google Translate’s Tap to Translate feature: Press and tap the home button to summon Now on Tap, and then select the “Translate this screen” option. Unlike Tap to Translate, though, Now on Tap doesn’t require you to highlight text — instead, it automatically parses the words and phrases for you. It supports translation of a handful of languages at launch, including English, French, German, Spanish, Russian, Portuguese, Italian, with more to join down the line.
Now on Tap’s other cool, new trick involves those familiar square graphics you’ve probably spotted on the sides of buildings, billboards, and bus stop banner ads: QR codes. Now on Tap can now extract whatever digital content (i.e., website link or photo) is hiding behind those opaque, monolithic black-and-white boxes. It’s a three-step process: launch your phone’s camera application, and, while centering the QR code in your viewfinder, simultaneously launch Now on Tap.
In addition, Now on Tap can now decipher the sorts of everyday bar codes you might encounter in a supermarket or convenience store. Scan a book’s code and you’ll get information about its publisher and author, or scan a bag of chips and you might get nutritional information. It’ll work on “packaged products, books, DVDs, and more,” Okonkwo said.
Finally, Now on Tap has gained a new section, Discover, that is intended to “help you learn more about things that interest you.” Basically, if you find yourself in the mood for internet dumpster diving, Discover will show you content tangentially related to the article, email, or conversation that is on your screen. If you’re reading an article about Jupiter, for instance, you’ll see relevant YouTube videos, photos, and news articles.
If a few of Now on Tap’s features sound a bit familiar, that is because many of them tread old ground. Google’s Goggles app, the search giant’s artificially intelligent camera app for Android and iOS, can also translate text, and recognize bar codes and QR codes. But the app’s unceremonious abandonment — Goggles hasn’t been updated since May 2014 — is perhaps a sign that Now on Tap will pick up the machine learning torch where it left off.
The update is rolling out to all users gradually, Okonkwo said. If you have the latest version of the Google app installed, you should see it shortly.
