Skip to main content

Pro photographers teach Google Clips when (and when not) to take a photo

lifelogging google clips press close
Image used with permission by copyright holder
How do you train an artificial intelligence program to choose when to take a picture? Using professional photographers of course. That’s part of the approach Google engineers took while developing Clips, the wearable camera that presses the shutter button for you (at least some of the time anyway). On the Google Design Blog, the company recently shared a glimpse at how the A.I. inside of the upcoming camera was built.

While a majority of A.I. programs are so large that cloud-based storage is a necessity, Google Clips houses the entire program inside of the camera, a security measure to keep content offline until the user decides to upload. Google engineers have spent three years building the camera — including the software.

Google says the approach to the Clips’ A.I. is human-centered. Google designer Josh Lovejoy put out a job ad for photographers, and in return was able to build a team that included a documentary filmmaker, a photojournalist, and a fine art photographer. After putting content from all those creatives together, the team asked, “What makes a memorable moment?”

While the group started with big ideas about rule of thirds, lighting and depth of field, the programmers began to realize they needed to simplify the list in order to teach those ideas to a computer — or as Lovejoy puts it, teaching Go, Dog, Go!  rather than starting with Shakespeare.

With a revised approach, the engineers began teaching the software using consistent examples — with each image fed into the system designed around teaching one specific concept. Many of those concepts, rather than what to look for, centered on what not to photograph. Clips was trained to ignore bouncing around inside of a purse, fingers over the lens, shaky movements, and blur, for example.

So besides what not to do, how did the system learn which moments to photograph? Clips is also trained in diversity — the camera is more likely to take a picture with a change in the environment, for example. Clips looks for visual changes using color along with avoiding allowing too much time to pass before taking another picture. As Google shared with the announcement of the lifelogging camera, the A.I. is also trained to learn which faces are familiar and which are strangers.

In the end, simplifying the program created the best results, the blog post suggests, alongside slight overshooting and giving the user the final say over which images to keep. Google also added a shutter button and software viewfinder to give the user the ability to snap shots themselves.

“In the context of subjectivity and personalization, perfection simply isn’t possible, and it really shouldn’t even be a goal,” Lovejoy said. “Unlike traditional software development, [machine learning] systems will never be ‘bug-free’ because prediction is an innately fuzzy science. But it’s precisely this fuzziness that makes ML so useful … success with Clips isn’t just about keeps, deletes, clicks, and edits (though those are important), it’s about authorship, co-learning, and adaptation over time.”

Google hasn’t yet shared an official launch date for Clips, though recent Federal Communications Commission approval suggests the release date could be coming up soon. Google has a sign-up list to receive updates on availability for the $250 life-logging camera.

Editors' Recommendations

Hillary K. Grigonis
Hillary never planned on becoming a photographer—and then she was handed a camera at her first writing job and she's been…
Apple iPhone 11 Pro camera guide: Take better photos with these tips
How to triple your photo fun with the iPhone 11 Pro's new three-lens camera
apple iphone 11 pro camera guide back

The iPhone 11 Pro is currently the best camera smartphone you can buy. Apple really took the fight to Google with its latest offering, showing it can still make incredible cameras to compete with the Pixel 3, the Huawei P30 Pro, and the many other great camera phones available today.

If you’ve purchased an iPhone 11 Pro, or are about to, here’s our guide to making the best use of the camera. Don’t forget to read our review of the iPhone 11 Pro, the iPhone 11, and the iPhone 11 Pro Max if you haven’t quite made up your mind about which new iPhone to buy.
Camera

Read more
Google Photos now lets you search for text in your images
google photos

Google Photos has long been the go-to cloud photo service for many, and it's now getting a new feature that makes it even better. The service already allows you to tag aspects of an image so that you can find images later and group them together, but now it'll allow you to search for photos based on text in those photos. So, for example, if you take a picture of a document, you'll be able to search for that photo later despite the image not being tagged with that text.

The new feature leverages Google's experience in artificial intelligence, and essentially allows Google Photos to scan your images for text. You can use that text too -- once you find the photo you're looking for, you can use the Lens button to copy the text, after which you can paste it into a word processor or other app to use later.

Read more
Google’s soccer-playing A.I. hopes to master the world’s most popular sport
A soccer ball in a net.

Think the player A.I. in FIFA ‘19 was something special? You haven’t seen anything yet! That’s because search giant Google is developing its own soccer-playing artificial intelligence. And, if the company’s history with machine intelligence is anything to go by, it’ll be something quite special.

In the abstract for a paper describing the work, the researchers note that: “Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly tested in a safe and reproducible manner. We introduce the Google Research Football Environment, a new reinforcement learning environment where agents are trained to play football in an advanced, physics-based 3D simulator.”

Read more