Skip to main content

Startup uses deep learning to let you shop for items by snapping photos

deep learning startup retail photos teaser 2
Image used with permission by copyright holder
Computer vision and image recognition, aka the ability to show a computer a picture of something and have it tell you what it is seeing, has been befuddling researchers for years.

In the past decade, however, it has taken enormous leaps forward thanks to deep learning neural networks. These are essentially vast computational approximations of the way that the human brain works, which can learn to recognize objects and people based on training examples.

A number of companies have attempted to use this kind of technology to transform the retail space, but so far none have really succeeded. We scan QR codes when we go into a store, or type the name of a book into Amazon, but the technology that lets us snap a quick picture of, say, a chair that we like and easily search for it (or similar items) online has largely eluded us.

This is a problem a team of Cornell University researchers are trying to solve. With a new startup called GrokStyle, co-founders and computer scientists Sean Bell and Kavita Bala have developed a state-­of-­the-­art algorithm that is more accurate than any competing system at recognizing objects within a picture and then linking them to real­-world items for sale.

Image used with permission by copyright holder

“What we’re focused on is the post­-search experience,” Bell, a computer science Ph.D., told Digital Trends. “That’s something I don’t feel has been very good so far. What we’re doing is not just recognizing a product, but finding out who else sells it, whether there are existing variations, whether you can get it in a different type of wood, etc. We’re not just trying to answer the ‘what is this?’ question about objects, but trying to answer the related questions people have in the shopping experience.”

As you can imagine, this isn’t easy ­– ­and particularly not since it does more than apps, which recognize, say, book covers and movies by letting you search for that item and that one alone.

“You may be in a restaurant and see a chandelier you like, and want to one to find one that is the same and available for sale — or similar, but in a different color or price point,” Bell said. “The idea is that you take a picture of something you like, and then based on that image, you get presented with a list of similar items. You can then filter these based on location, material, and various other metrics. It’s not just simply about either finding something or not finding it; we wanted to give you a wide variety of choice.”

Going forward, the idea is to even let users single out particular qualities of an object (say, its grain of wood or fabric) and then find other complimentary items. “Our system lets you hold certain aspects as constants, and then change other attributes,” he said.

The system the team has developed, described in the journal ACM Transactions on Graphics under the name “Learning visual similarity for product design with convolutional neural networks,” was benchmarked to be an astonishing two times as accurate as competing methods. It is hoped that users will be able to try out the system over the coming months.

“This is an exciting area, and ultimately we think it’ll come down to who has the best technology,” Bell said. “Right now we’re state of the art in terms of product image recognition. We’ve demonstrated to the academic community that our system for doing this is the most accurate. We want to continue that edge going forwards.”

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
Meet the game-changing pitching robot that can perfectly mimic any human throw
baseball hitter swings and misses

Who’s your favorite baseball pitcher? Shane McClanahan? Sandy Alcantara? Justin Verlander? Whoever you said, two of the top sports-tech companies in the U.S. -- Rapsodo and Trajekt Sports -- have teamed up to build a robot version of them, and the results are reportedly uncannily accurate.

Okay, so we’re not talking about walking-talking-pitching standalone robots, as great a sci-fi-tinged MLB ad as that would be. However, Rapsodo and Trajekt have combined their considerable powers to throw a slew of different technologies at the problem of building a machine that's able to accurately simulate the pitching style of whichever player you want to practice batting against -- and they may just have pulled it off, too.

Read more
The best portable power stations
EcoFlow DELTA 2 on table at campsite for quick charging.

Affordable and efficient portable power is a necessity these days, keeping our electronic devices operational while on the go. But there are literally dozens of options to choose from, making it abundantly difficult to decide which mobile charging solution is best for you. We've sorted through countless portable power options and came up with six of the best portable power stations to keep your smartphones, tablets, laptops, and other gadgets functioning while living off the grid.
The best overall: Jackery Explorer 1000

Jackery has been a mainstay in the portable power market for several years, and today, the company continues to set the standard. With three AC outlets, two USB-A, and two USB-C plugs, you'll have plenty of options for keeping your gadgets charged.

Read more
CES 2023: HD Hyundai’s Avikus is an A.I. for autonomous boat and marine navigation
Demonstration of NeuBoat level 2 autonomous navigation system at the Fort Lauderdale International Boat Show

This content was produced in partnership with HD Hyundai.
Autonomous vehicle navigation technology is certainly nothing new and has been in the works for the better part of a decade at this point. But one of the most common forms we see and hear about is the type used to control steering in road-based vehicles. That's not the only place where technology can make a huge difference. Autonomous driving systems can offer incredible benefits to boats and marine vehicles, too, which is precisely why HD Hyundai has unveiled its Avikus AI technology -- for marine and watercraft vehicles.

More recently, HD Hyundai participated in the Fort Lauderdale International Boat Show, to demo its NeuBoat level 2 autonomous navigation system for recreational boats. The name mashes together the words "neuron" and "boat" and is quite fitting since the Avikus' A.I. navigation tech is a core component of the solution, it will handle self-recognition, real-time decisions, and controls when on the water. Of course, there are a lot of things happening behind the scenes with HD Hyundai's autonomous navigation solution, which we'll dive into below -- HD Hyundai will also be introducing more about the tech at CES 2023.

Read more