Startup uses deep learning to let you shop for items by snapping photos

deep learning startup retail photos teaser 2
Computer vision and image recognition, aka the ability to show a computer a picture of something and have it tell you what it is seeing, has been befuddling researchers for years.

In the past decade, however, it has taken enormous leaps forward thanks to deep learning neural networks. These are essentially vast computational approximations of the way that the human brain works, which can learn to recognize objects and people based on training examples.

A number of companies have attempted to use this kind of technology to transform the retail space, but so far none have really succeeded. We scan QR codes when we go into a store, or type the name of a book into Amazon, but the technology that lets us snap a quick picture of, say, a chair that we like and easily search for it (or similar items) online has largely eluded us.

This is a problem a team of Cornell University researchers are trying to solve. With a new startup called GrokStyle, co-founders and computer scientists Sean Bell and Kavita Bala have developed a state-­of-­the-­art algorithm that is more accurate than any competing system at recognizing objects within a picture and then linking them to real­-world items for sale.


“What we’re focused on is the post­-search experience,” Bell, a computer science Ph.D., told Digital Trends. “That’s something I don’t feel has been very good so far. What we’re doing is not just recognizing a product, but finding out who else sells it, whether there are existing variations, whether you can get it in a different type of wood, etc. We’re not just trying to answer the ‘what is this?’ question about objects, but trying to answer the related questions people have in the shopping experience.”

As you can imagine, this isn’t easy ­– ­and particularly not since it does more than apps, which recognize, say, book covers and movies by letting you search for that item and that one alone.

“You may be in a restaurant and see a chandelier you like, and want to one to find one that is the same and available for sale — or similar, but in a different color or price point,” Bell said. “The idea is that you take a picture of something you like, and then based on that image, you get presented with a list of similar items. You can then filter these based on location, material, and various other metrics. It’s not just simply about either finding something or not finding it; we wanted to give you a wide variety of choice.”

Going forward, the idea is to even let users single out particular qualities of an object (say, its grain of wood or fabric) and then find other complimentary items. “Our system lets you hold certain aspects as constants, and then change other attributes,” he said.

The system the team has developed, described in the journal ACM Transactions on Graphics under the name “Learning visual similarity for product design with convolutional neural networks,” was benchmarked to be an astonishing two times as accurate as competing methods. It is hoped that users will be able to try out the system over the coming months.

“This is an exciting area, and ultimately we think it’ll come down to who has the best technology,” Bell said. “Right now we’re state of the art in terms of product image recognition. We’ve demonstrated to the academic community that our system for doing this is the most accurate. We want to continue that edge going forwards.”

Emerging Tech

Awesome Tech You Can’t Buy Yet: inflatable backpacks and robotic submarines

Check out our roundup of the best new crowdfunding projects and product announcements that hit the Web this week. You can't buy this stuff yet, but it sure is fun to gawk!

From white skies to blurry subjects, how to fix common photo mistakes

You snapped the photo at the perfect moment, but the image is blurry. Understanding the most common photography mistakes can help capture better memories and more likable Instagrams. Here's how to fix these seven common photography…

When you're ready to shoot seriously, these are the best DSLRs you can buy

For many photographers the DSLR is the go-to camera. With large selection of lenses, great low-light performance, and battery endurance, these DSLRs deliver terrific image quality for stills and videos.

These 30 apps are absolutely essential for Mac lovers

There are literally hundreds of thousands of great software programs compatible with MacOS, but which should you download? Look no further than our list of the best Mac apps you can find for the latest MacOS and how they can help out your…
Emerging Tech

Watch as a ‘lifeguard drone’ rescues a swimmer struggling at sea

These days, drones are finding a range of roles in a myriad of fields. Lifeguards, for example, are making use of the drone's ability to quickly deploy flotation devices while also offering an eye in the sky to survey the scene.
Emerging Tech

Wish you could fly? Here are the best drones on the market right now

To help you navigate the increasingly large and ever-changing landscape of consumer UAVs, here's a no-nonsense rundown of the best drones you can buy right now

Art-inspired face blurring can obscure identity without losing humanity

Researchers have developed an AI-generated anonymity system that “paints” over video frames, using inspiration from masters like Picasso and Van Gogh to reimagine a person’s appearance. The goal is to minimize outer resemblance but…
Emerging Tech

The best drone photos from around the world

Most of today's drones come equipped with high-end cameras, which are quickly revolutionizing the world of aerial photography as we know it. Here are some of the best drone photos from around the world.
Emerging Tech

Experiment suggests that the best robot bosses could be jerks

Researchers have been investigating how future robot bosses can coax the most productivity out of us flesh-and-blood employees. The sad answer? Quite possibly by behaving like jerks.
Emerging Tech

VR experience shows caregivers what it’s like to live with Alzheimer’s disease

Los Angeles-based VR startup Embodied Labs has developed a virtual experience that puts users in the shoes of a person with Alzheimer’s disease, the most common type of dementia in the U.S.
Emerging Tech

Welcome to the uncanny valley: This robot head shows lifelike expressions

SEER is a robot head that is capable of recognizing the facial expressions of the people that it interacts with, and then mirroring their same expression back at them. Check it out.

Crypto-intrigued? Here's how to buy Bitcoin for the first time

Is it time to purchase your first Bitcoin investment? If you're ready to get involved in the cryptocurrency, we'll walk you through how to pick an exchange, how to choose the right wallet, and how to buy Bitcoin the safe way!
Emerging Tech

6 flying cars that you might actually be able to own (and fly) in your lifetime

Just like cellphones used to be only for the rich, soon taking a personal flying machine to work is going to be a part of everyday life. Here are six pieces of tech making that possible.
Emerging Tech

Spectacular twisting tower aims to become Australia’s tallest building

A spectacular twisting structure bedecked with trees and plants is aiming to become Australia's tallest building. The "Green Spine" was recently announced as the winner of an international design competition.