The feature starts with a search using at least one keyword, but then within those results, clicking on the three dots in the upper right corner of a photo will bring up the option to “search for similar photos.” One more tap and the search fills with similar shots, including aspects that are difficult to describe in keywords like texture, patterns, and even where the subject is in the frame.
The feature allows the system to search beyond just written tags and object recognition by taking data from the image itself. For example, if you are searching for a photo of an orange tabby cat, many photographers (or Flickr’s automatic object recognition) probably only used the cat tag, not the specific breed or color. Since the similar search considers things like the cat’s colors and stripes, you’ll likely find a bigger variety of tabby cats to choose from if you run the similar search on a tabby from your initial search.
The system isn’t perfect — our search for an orange tabby also turned up a similar colored terrier — but overall aims to both simplify searches and branch off into potentially inspiring yet unintended directions.
AI isn’t new to Flickr’s image search — aspects like object recognition have long helped to boost the search beyond manually added tags. The issue is that the object recognition tosses a lot of information, such as where that object is, what color it is, and even the textures it has. Instead, the updated neural network grabs that information ahead of time and converts those details into sets of numbers that tell the computer things like color, shape, and position.
The computer compares that set of numbers to another image’s set of numbers, and the numbers that are the closest together help determine which images are the most similar. Flickr also created a locally optimized algorithm that essentially filters an image’s rankings to the highest priorities. Otherwise, the amount of data associated with a photo would translate into longer load times instead of a quicker, more accurate search tool.
The similarity search joins Flickr’s keyword search bar along with its color, pattern, and orientation filters.
