Startup uses deep learning to let you shop for items by snapping photos

deep learning startup retail photos teaser 2
Computer vision and image recognition, aka the ability to show a computer a picture of something and have it tell you what it is seeing, has been befuddling researchers for years.

In the past decade, however, it has taken enormous leaps forward thanks to deep learning neural networks. These are essentially vast computational approximations of the way that the human brain works, which can learn to recognize objects and people based on training examples.

A number of companies have attempted to use this kind of technology to transform the retail space, but so far none have really succeeded. We scan QR codes when we go into a store, or type the name of a book into Amazon, but the technology that lets us snap a quick picture of, say, a chair that we like and easily search for it (or similar items) online has largely eluded us.

This is a problem a team of Cornell University researchers are trying to solve. With a new startup called GrokStyle, co-founders and computer scientists Sean Bell and Kavita Bala have developed a state-­of-­the-­art algorithm that is more accurate than any competing system at recognizing objects within a picture and then linking them to real­-world items for sale.


“What we’re focused on is the post­-search experience,” Bell, a computer science Ph.D., told Digital Trends. “That’s something I don’t feel has been very good so far. What we’re doing is not just recognizing a product, but finding out who else sells it, whether there are existing variations, whether you can get it in a different type of wood, etc. We’re not just trying to answer the ‘what is this?’ question about objects, but trying to answer the related questions people have in the shopping experience.”

As you can imagine, this isn’t easy ­– ­and particularly not since it does more than apps, which recognize, say, book covers and movies by letting you search for that item and that one alone.

“You may be in a restaurant and see a chandelier you like, and want to one to find one that is the same and available for sale — or similar, but in a different color or price point,” Bell said. “The idea is that you take a picture of something you like, and then based on that image, you get presented with a list of similar items. You can then filter these based on location, material, and various other metrics. It’s not just simply about either finding something or not finding it; we wanted to give you a wide variety of choice.”

Going forward, the idea is to even let users single out particular qualities of an object (say, its grain of wood or fabric) and then find other complimentary items. “Our system lets you hold certain aspects as constants, and then change other attributes,” he said.

The system the team has developed, described in the journal ACM Transactions on Graphics under the name “Learning visual similarity for product design with convolutional neural networks,” was benchmarked to be an astonishing two times as accurate as competing methods. It is hoped that users will be able to try out the system over the coming months.

“This is an exciting area, and ultimately we think it’ll come down to who has the best technology,” Bell said. “Right now we’re state of the art in terms of product image recognition. We’ve demonstrated to the academic community that our system for doing this is the most accurate. We want to continue that edge going forwards.”

Movies & TV

Oscar-winning FX master explains why ‘First Man’ is a giant leap for filmmaking

Paul Lambert, the Oscar-winning visual effects supervisor on First Man, reveals the innovative techniques that blended old footage with modern movie magic to make the Apollo 11 mission to the moon resonate with audiences 50 years later.
Home Theater

From the Roku Ultra to the Fire TV Cube, these are the best streaming devices

There are more options for media streamers than ever, so it’s more difficult to pick the best option. But that’s why we're here. Our curated list of the best streaming devices will get you online in no time.

Is this the first image of a Galaxy S10 being used in real life?

It won't be long now; With 2019 underway, the Samsung Galaxy S10 is almost here. Before it arrives, here's absolutely everything you need to know about all three of Samsung's next flagships.
Emerging Tech

Face-scanning A.I. can help doctors spot unusual genetic disorders

Facial recognition can unlock your phone. Could it also be used to identify whether a person has a rare genetic disorder, based on their facial features? New research suggests it can.
Emerging Tech

Want to know which drones are flying near you? There’s an app for that

Want to know what that mysterious drone buzzing over your head is up to? A new system developed by AirMap, Google Wing, and could soon tell you -- via a map on your phone.
Emerging Tech

A Japanese hotel fires half its robot staff for being bad at their jobs

Japan’s oddball Henn na Hotel has fired half of its 243 robot staff. The reason? Because these labor-saving machines turned out to be causing way more problems than they were solving.
Emerging Tech

CERN plans to build a massive particle collider that dwarfs the LHC

CERN already has the world's biggest particle accelerator. Now it wants a bigger one. Meet the 9 billion euro Future Circular Collider that will allow physicists to extend their study of the universe and matter at the smallest level.
Emerging Tech

Forget fireworks. Japan will soon have artificial meteor showers on tap

Tokyo-based startup Astro Live Experiences is preparing to launch its first artificial meteor shower over Japan, serving as a showcase of its prowess in the space entertainment sector.

Robomart’s self-driving grocery store is like Amazon Go on wheels

Robomart's driverless vehicle is like an Amazon Go store on wheels, with sensors tracking what you grab from the shelves. If you don't want to shop online or visit the grocery store yourself, Robomart will bring the store to you.
Emerging Tech

Glowing space billboards could show ads in the night sky

Look up at the night sky in 2020 and you might see an ad for McDonald's floating among the stars. A Russian startup is working on a project that uses a constellation of small satellites in low-Earth orbit to create glowing ads.
Emerging Tech

New brainwave reader tells teachers if students are concentrating

Massachusetts-based startup BrainCo has developed brainwave-reading headbands which can reportedly help reveal if students are concentrating in class. Here's how they're being used.
Emerging Tech

Fears about kids’ screen use may have been overblown, Oxford researchers find

Many people take it as gospel that digital technologies are harmful to young people’s mental health. But is this true? A recent study from the University of Oxford takes a closer look.
Emerging Tech

Meet Wiliot, a battery-less Bluetooth chip that pulls power from thin air

A tiny chip from a semiconductor company called Wiliot could harvest energy out of thin air, the company claims. No battery needed. The paper-thin device pulls power from ambient radio frequencies like Wi-Fi, Bluetooth, and cell signals.
Emerging Tech

Hexbot is a modular robot arm that does everything from drawing to playing chess

Who wouldn’t want their own personal robot arm to do everything from laser engraving to competing against you in a game of chess? That's what Hexbot, a new modular robot, promises to deliver.