Skip to main content

Hot dog! Yelp photo classification makes food shots easier to browse

yelp photo classification sandwiches somacrop
A picture may be worth a thousand words — but it can also be worth lunch. Yelp visitors often rely on photographs to determine their next dining spot, and now, updated artificial intelligence is helping to both prioritize the best images and categorize each shot. So how can a computer determine a salad from a slice of pizza? Lindsay Goyne, Yelp consumer product manager Lindsay Goyne and software engineer Alex Miller recently sat down with Digital Trends to share just how new software is pushing more relevant photos to foodies through new Yelp photo classification.


Unlike the new app inspired from HBO’s Silicon Valley, which doesn’t actually have much real world use outside of a few laughs, Yelp’s new hot dog-identifying app works to create a better experience for Yelp users. The neural network identifies over 20 different types of food, categorizing each dish so users can find out what that new diner’s burger looks like without digging through a few hundred photos. Hungry for pizza? Tap the photos, the food option, then click the pizza tab and see if that restaurant you’ve always wanted to try puts big chunks of tomato on top or whether they have deep dish.

“Photos have been trending up through multiple photo apps and the web,” said Goyne, “it’s a new favorite way to consume information. For some of these lower-risk options, like finding an option for lunch where you don’t need to do a bunch of research and you aren’t spending much money, photos are a great way to do that research.”

The software, launching earlier this year, is a result of user feedback. Yelp said that, when it comes to restaurant reviews, some of the most common questions were whether they had a certain dish. By organizing photos uploaded from other users, Yelp is able to present that information in a more precise way.


The convolutional neural network was based on a model built by Google researchers, Miller said. Artificial intelligence platforms are frequently shared among developers so new programs don’t need to start from scratch — like Google’s Wi-Fi-free object recognition algorithm. Yelp gave the system expanded object recognition capabilities by feeding the system thousands of images for each food category. With that data, the computer was able to learn what makes a hot dog different from a sandwich.

Along with the food categories, images are automatically sorted by shots of the interior and exterior and photos of the menu.

Yelp also uses both user response and artificial intelligence to determine which photographs to show first. While the company is continuing to develop software for ordering the photos, the software mixes user feedback — including clicks on “helpful” and “like” — and mixes that with an AI program that puts blurry, dark photos toward the end of the list and images with a nice depth of field and more contrast to the top of the list. The program also tries to prevent similar photos from showing up right next to each other, like three burgers in a row, and determines what image to use as the main photo.


Yelp’s software engineers are also working to teach the computer to recognize what the most important object in the photograph is, so that when the images is cropped to fit on different areas of the web page, the subject is still included.

The software is helping users sift through review photos that, on the most popular restaurants, can number over 1,000. Users won’t see the feature universally, since locations with only a few photographs do not integrate the category organization.

Yelp plans to bring the image classification system into more areas and is currently researching how a similar feature could help users looking for home services and automotive.

Hillary K. Grigonis
Hillary never planned on becoming a photographer—and then she was handed a camera at her first writing job and she's been…
Best camera deals: Save on DSLR, action cameras, point-and-shoot
Best DSLR cameras for beginners Canon Rebel T7i

There are a lot of great camera phones that offer high-quality pictures and video, but if you really want to take your photography to the next level, then you'll want to grab yourself a self-contained camera. Not only do they have more space for complex hardware, but many cameras have interchangeable lenses, so you can do different types of photography. Luckily, there are a lot of options to pick from, including Mirrorless vs. DSLR, and we've collected some of our favorite camera deals below.

That said, if you'd like something a little bit different, check out these drone deals, and for more action cameras, there are these GoPro deals, too.
Best DSLR camera deals

Read more
This Canon 4K camera has a $150 price cut at Best Buy
The Canon EOS R50 4K Mirrorless Camera.

If you’ve been on the lookout for camera deals, we’ve got an amazing one to share! Best Buy is selling the Canon EOS R50 Mirrorless Camera for $650 this week. That’s a $150 markdown from its original $800 price! For decades, Canon has been one of the most respected brands in the consumer tech space, and its cameras are some of the best in the business. 

Why you should buy the Canon EOS R50 camera
There’s a few core components running the show here: The 24.2-megapixel CMOS Sensor is the R50’s powerful processor. There are plenty of variables at play when snapping pictures, and Canon’s CMOS ensures that you’ll always get the best-looking image possible; even when shooting in light-starved environments. But a camera is nothing without lenses, and the R50 comes with the Canon RF-S 18-45mm Standard Zoom. Unobtrusive and easy to toss in a camera bag, you’ll be able to take the RF-S wherever you and your camera body venture. 

Read more
Best drone deals: Get a cheap drone for $47 and more
The DJI Mini 3 Pro in flight with spring flowers in the background.

If you really enjoy the outdoors and filming your adventures, then there's really nothing like a Drone to capture some incredible footage, and while action cameras are great, you can't deny the effect of aerial photography. Of course, there are a lot of different types of drones out there depending on your needs, and a lot of them can be pretty expensive; after all, not everybody can afford or needs a high-end DJI. That's why a lot of folks go for DJI alternatives and other deals, which is why we've collected some of our favorites below, ranging from some great budget options to great bundles that will take things to the next level.

Of course, if you prefer a more traditional experience, you could always check out these GoPro deals and camera deals instead.
Radclo Mini Drone -- $42, was $63

Read more