Pixel 2 owners get the first glimpse of Google Lens computer vision possibilities

Unveiled earlier in 2017 at Google I/O, the first public version of the artificially intelligent computer vision program Google Lens is now part of the new Google Pixel 2. During Wednesday’s October 4 event in San Fransisco, Google shared a preview of Lens that will ship inside the new Pixel 2 smartphone, with integration into both Google Photos and Google Assistant.

Google Lens is the tech giant’s computer vision software that collects information from a photograph to either save some time by skipping the typing or to learn something new about the things that we see around us. The tool effectively mixes Google search with a camera, and while the Pixel 2 only contains a preview of the feature, the platform already creates a few promising shortcuts.

During the event, Google’s Aparna Chennapragada shared how the new feature allows the smartphone’s camera to be used as a sort of keyboard. When taking a photo of something with text, like a flyer, Google Lens allows users to highlight text such as email addresses, phone numbers, websites, and street addresses and copy the information. The shortcut makes it easy to look up a location on Google Maps or call a phone number without typing it into the keyboard.

Besides serving as a visual shortcut to typing in long and unusual email addresses, Google Lens is also designed to help users understand the objects they see — starting with art and entertainment. Snapping a photo of a piece of art will lead to who the artist is and what else they painted. See a movie poster? Lens will tell you if the flick is worth watching or not. Snapping photos of album covers and book covers also lead you to more details on the work.

The preview inside the Pixel 2 is just a start for the computer vision software. When the software was first announced, Google listed a long number of possibilities, including translating text, getting more details on a business, reading Wi-Fi network settings or learning the name of that flower you just spotted.

Google’s computer vision also works with existing photos, powering a number of tools inside the native Google Photos app on the Pixel 2. Searching for specific objects, people and even famous landmarks is possible through the program’s auto-tagging feature.

Google Lens is based on machine learning — Google essentially used those millions of photos in their search results to train the computer to recognize what a specific object looks like. With enough photos, the program can learn to recognize what the Eiffel Tower looks like on a cloudy day, lit up at night or even blurred from camera shake to correctly identify what is in the photo.

Chennapragada said that Google Lens will continue to improve with use. For example, she said, Google’s voice recognition, at first, wouldn’t always recognize speech correctly, particularly with factors like accents. Now, after several years of development, Google voice has a 95 percent accuracy rate.

Google CEO Sudar Pichai said that the object recognition AI built by Google had a 39 percent accuracy rate. Using what’s called AutoML, which is essentially artificial intelligence building more AI programs, that accuracy rate has improved to 43 percent and is continuing to improve.

“This is why we are excited about the shift from mobile first to AI first, it’s radically rethinking how computers work,” Pichai said during the presentation. “Computers should adapt to how people live their life, rather than people adapting to computers.”

Google Lens will first be available in Pixel 2 by tapping the Lens icon inside both Google Photos and Google Assistant.

Emerging Tech

Google’s radar-sensing tech could make any object smart

Computer scientists have shown how Google’s Soli sensor can be used to make dumb objects smart. Here's why radar-powered computing could finally make the dream of smart homes a reality.
Smart Home

Project Alias is a ‘smart parasite’ that stops smart speakers from listening

Two designers chose to do something about nosy smart speakers. The result is Project Alias, a "smart parasite" that whispers nonsense to Google Home and Alexa until it hears a specific wake word.
Computing

Chromebook 13 vs. Google Pixelbook: Acer model takes on the king

Acer's Chromebook 13 is throwing tons of speed at the Chrome OS market, to go with a midrange build and traditional clamshell design. Is that enough to challenge the Google Pixelbook?
Mobile

Benchmark scores surface for Google's midrange Pixel 3 XL Lite

The Google Pixel 3 and Pixel 3 XL are considered to be two of the best Android smartphones, but it looks like Google could be prepping a midrange line. Say hello to the Pixel 3 Lite and Pixel 3 Lite XL.
Wearables

The best Apple Watch bands and straps to stylize your timepiece

If you have an Apple Watch, you know how easy it is to take off the strap it came with, so why not buy yourself another one? Here, we've gathered the best Apple Watch bands we've seen so far. There's something for everyone.
Mobile

How to choose an iPad in 2019: A practical guide to Apple’s tablets

Selecting an iPad from Apple's lineup can be intimidating, but it doesn't have to be. Our comprehensive guide should put the numbers and specs in practical, easy-to-understand terms. Find your ideal iPad with the help of our guide.
Mobile

The Cat S48c is the phone designed for construction workers (or the clumsy)

The Cat S48c is a rugged smartphone that's available from Sprint. It mixes midrange specs with a huge battery wrapped in an extremely tough and protective body. If you need a phone that can survive the construction site, then this is it.
Mobile

Apple resurrects the iPhone SE with brand-new units starting at $249

Apple quietly started selling the iPhone SE again, at even lower prices than when it was discontinued four months ago. Brand new units of the 32GB version are on sale for $249, while the 128GB version is going for $299.
Android

Popular Android navigation apps are just Google Maps with ads, researcher says

A malware researcher found that 19 free Android navigation apps on the Google Play Store were nothing more than Google Maps, but with ads. One of the apps asked for a payment to remove the ads, while some of them presented security risks.
Mobile

Google Maps will now help drivers stay within speed limits, avoid speed traps

Google Maps will now start showing speed limits and speed camera locations, so that drivers will not be flagged for speeding tickets. The new features arrive to the app years after they were introduced in Waze.
Mobile

Text up a storm with the best messaging apps for iOS and Android

These days, most people tend to favor digital messages over phone calls. We have the best messaging apps that allow you to share photos and documents, send text messages, and more with end-to-end encryption.
Mobile

If you want Samsung's advanced folding phone, be prepared to pay a lot for it

Samsung has been showcasing bendable display tech for a few years and now a folding smartphone might finally arrive. The Galaxy X, or perhaps the Galaxy Fold, may be the company's first example. Here's everything we know about it.
Mobile

Do these case images confirm a side-mounted fingerprint scanner on the S10 E?

It won't be long now; With 2019 underway, the Samsung Galaxy S10 is almost here. Before it arrives, here's absolutely everything you need to know about all three of Samsung's next flagships.
Mobile

T-Mobile 5G rollout: Here is everything you need to know

2019 will be a huge year for T-Mobile. Not only is a merger with Sprint likely, but T-Mobile is also in the midst of building out its next-generation mobile service. Here's everything you need to know about the T-Mobile 5G rollout.