Hot dog! Yelp photo classification makes food shots easier to browse

yelp photo classification sandwiches somacrop
A picture may be worth a thousand words — but it can also be worth lunch. Yelp visitors often rely on photographs to determine their next dining spot, and now, updated artificial intelligence is helping to both prioritize the best images and categorize each shot. So how can a computer determine a salad from a slice of pizza? Lindsay Goyne, Yelp consumer product manager Lindsay Goyne and software engineer Alex Miller recently sat down with Digital Trends to share just how new software is pushing more relevant photos to foodies through new Yelp photo classification.

yelp photo classification sandwichessomamobile

Unlike the new app inspired from HBO’s Silicon Valley, which doesn’t actually have much real world use outside of a few laughs, Yelp’s new hot dog-identifying app works to create a better experience for Yelp users. The neural network identifies over 20 different types of food, categorizing each dish so users can find out what that new diner’s burger looks like without digging through a few hundred photos. Hungry for pizza? Tap the photos, the food option, then click the pizza tab and see if that restaurant you’ve always wanted to try puts big chunks of tomato on top or whether they have deep dish.

“Photos have been trending up through multiple photo apps and the web,” said Goyne, “it’s a new favorite way to consume information. For some of these lower-risk options, like finding an option for lunch where you don’t need to do a bunch of research and you aren’t spending much money, photos are a great way to do that research.”

The software, launching earlier this year, is a result of user feedback. Yelp said that, when it comes to restaurant reviews, some of the most common questions were whether they had a certain dish. By organizing photos uploaded from other users, Yelp is able to present that information in a more precise way.


The convolutional neural network was based on a model built by Google researchers, Miller said. Artificial intelligence platforms are frequently shared among developers so new programs don’t need to start from scratch — like Google’s Wi-Fi-free object recognition algorithm. Yelp gave the system expanded object recognition capabilities by feeding the system thousands of images for each food category. With that data, the computer was able to learn what makes a hot dog different from a sandwich.

Along with the food categories, images are automatically sorted by shots of the interior and exterior and photos of the menu.

Yelp also uses both user response and artificial intelligence to determine which photographs to show first. While the company is continuing to develop software for ordering the photos, the software mixes user feedback — including clicks on “helpful” and “like” — and mixes that with an AI program that puts blurry, dark photos toward the end of the list and images with a nice depth of field and more contrast to the top of the list. The program also tries to prevent similar photos from showing up right next to each other, like three burgers in a row, and determines what image to use as the main photo.

yelp photo classification sandwiches soma copy

Yelp’s software engineers are also working to teach the computer to recognize what the most important object in the photograph is, so that when the images is cropped to fit on different areas of the web page, the subject is still included.

The software is helping users sift through review photos that, on the most popular restaurants, can number over 1,000. Users won’t see the feature universally, since locations with only a few photographs do not integrate the category organization.

Yelp plans to bring the image classification system into more areas and is currently researching how a similar feature could help users looking for home services and automotive.


The Teslasuit could turn Black Mirror’s terrifying ‘Playtest’ into a reality

We spoke with Teslasuit co-founder Dimitri Mikhalchuk about VR gaming at CES 2019. With all its features, the future of the Teslasuit and virtual reality look bright. And it also sounds a bit like a Black Mirror episode.
Social Media

Nearly 75 percent of U.S. users don’t realize Facebook tracks their interests

Did you know Facebook tracks your interests, including political and multicultural affiliations? According to a recent Pew study, 74 percent of adult users in the U.S. have no idea Facebook keeps a running list of your interests.

We tried all the latest and greatest smartphones to find the best of 2019

Smartphones are perhaps the most important and personal piece of tech on the planet. That’s why it’s important to pick the best phone for your individual needs. Here are the best smartphones you can buy.

Benchmark results show Snapdragon 855 destroys previous-generation chip

Almost exactly a year after the launch of the Snapdragon 845, Qualcomm took the wraps off of its next-generation mobile platform, the new Snapdragon 855. The new chip puts an emphasis on A.I. performance.

Going somewhere? Capture more than your phone can with the best travel cams

Hitting the road or doing some globetrotting this year? Bring along the right camera to capture those once-in-a-lifetime vacation memories. Here's a list of some of our current favorites.

From 4K powerhouses to tiny action cams, here are the best video cameras

Although not as popular as they once were, dedicated video cameras still have their benefits. From travel vlogging to home movies to recording your kid's little league game, here are the best video cameras you can buy right now.

Authentic, holistic, retro photography is in: Here are 2019’s predicted trends

What types of imagery are we most drawn to? According to recent stock photography data from Adobe, StoryBlocks, and Shutterstock, authentic, holistic, and humanitarian content will be in high demand in 2019.

Starting your very own vlog? Here are the best cameras to buy

Any camera that shoots video can be used to vlog, but a few models stand out from the crowd thanks to superior image quality, ergonomics, and usability. When it comes to putting your life on YouTube, here are the best cameras for the job.

The best mirrorless cameras pack all the power of a DSLR, minus the bulk

Mirrorless cameras offer a lot of photography firepower, inside a compact body. Explore the best mirrorless cameras, from the pro-level to the beginner-friendly shooters, in this guide.

This A.I.-powered camera follows the action to produce epic selfie videos

Want to capture more epic action selfies? The Obsbot Tail is a camera-gimbal combo that uses artificial intelligence to follow the action. Using a handful of different modes, the camera works to keep the action in the frame.

Sony crams its best camera tech into the new $900 A6400

Love Sony's autofocus, but can't stomach the full-frame price? The Sony A6400 mirrorless camera uses some of the same autofocus technology and the processor of the A9 in a compact, more affordable crop-sensor camera.

These point-and-shoot cameras make your smartphone pics look like cave paintings

If your smartphone camera just isn't giving you the results you're looking for, maybe it's time to step up your game. The latest and greatest point-and-shoot cameras offer large sensors, tough bodies, and long lenses -- something no phone…

GoPro bumps resolution on Fusion 360 cam to 5.6K with new firmware

Currently available in public beta, Fusion firmware version 2.0 offers a new 5.8K mode that results in 5.6K output when the 360 camera's two hemispheres are stitched together. It also adds support for 24 fps video and RAW time-lapse…

With 5-stop optical stabilization, Fujifilm GF 100-200mm is ready for adventure

Fujifilm revealed a new lens designed to deliver on the GFX system's promise of adventure-ready medium-format photography. The GF 100-200mm F5.6 R is a weather resistant, relatively lightweight, 2x telephoto with impressive stabilization.