Forget keywords — this new system lets you search with rudimentary sketches

Could the future of online shopping be as simple as sketching out what you’re looking for and letting a computer figure out the rest? That’s the working theory of two researchers at Queen Mary University of London’s (QMUL) School of Electronic Engineering and Computer Science. They’ve taught a deep learning neural network — an incredibly powerful tool that mimics the way that the human brain works — to recognize hand-drawn sketches and use them to search for real-life products.

The network was “trained” to match sketches to photos based on a data set consisting of around 30,000 sketch-photo comparisons. From these it learned to interpret the way that people depict real objects in hand drawing. Most impressive of all is the fact that the sketches drawn by users don’t even have to be all that detailed — but the more detail users do add, the more accurate the search results become.

“I would hope that within six months you’ll be able to log into a shopping site and use this technology.”

“Sketch is a very flexible tool,” co-developer Yi-Zhe Song, director of QMUL’s SketchX Research Lab, told Digital Trends. “For example, you might see a pair of shoes that you like, but not approve of every detail of them — such as the buckle or the heel. In that case, you can draw the shoe you would like to see — and then let the computer search for the closest possible match.”

But as impressive a computer science project as this is, why would anyone want to spend time drawing pictures instead of typing a quick text-based search in, or snapping a picture as companies like Amazon now allow you to do? Song has an answer.

“Both of those search modalities have problems,” he says. “Text-based search means that you have to try and describe the item you are looking for. This is especially difficult when you want to describe something at length, because retrieval becomes less accurate the more text you type. Photo-based search, on the other hand, lets you take a picture of an item and then find that particular product. It’s very direct, but it is also overly constrained, allowing you to find just one specific product instead of offering other similar items you may also be interested in.”

While Song and his colleague Timothy Hospedales, director of QMUL’s Applied Machine Learning Lab, acknowledge that both of these search methods have their use-cases, and they think the time is right for them to be joined by sketch-based search.

“Today, almost all of us have smartphones, tablets and even smartwatches with touchscreens you can draw on,” Song notes. “This is something that was relatively rare just 10 years ago.”

Song and Hospedales recently showed off their new creation to an audience at the International Conference on Computer Vision and Pattern Recognition in Las Vegas, and are now talking with retailers about incorporating the product into the online marketplace. “I would hope that within six months you’ll be able to log into a shopping site and use this technology,” Hospedales told Digital Trends.

For the sake of easy shopping, we hope so too.

Product Review

The iPad Pro is the best tablet ever. But don't sell your laptop just yet

Apple has unveiled a big redesign for the iPad Pro, slimming down the bezels, adding Face ID, and the ability to attach and charge the Apple Pencil. All of this comes at a high cost however, as the iPad Pro starts at $799.
Mobile

Apple's iOS 12.1.1 makes it easier to switch cameras in FaceTime

After months of betas, the final version of iOS 12 is here to download. The latest OS comes along with tons of new capabilities, from grouped notifications to Siri Shortcuts. Here are all the features you'll find in iOS 12.
Product Review

Meet Z6, the breakout star in Nikon's new mirrorless lineup

The Nikon Z6 is the sibling to the new mirrorless Z7 -- but for some photographers, the cheaper Z6 may be the better option. Read where the $2,000 camera beats the $3,400 one (and where it doesn’t) in our Nikon Z6 review.
Emerging Tech

We’re going to the Red Planet! All the past, present, and future missions to Mars

SpaceX isn't the only organization pining to visit the Red Planet. Here's a detailed list of all operational and planned missions to Mars, along with explanations of their objectives, spacecraft details, and mission proposals.
Emerging Tech

There’s a giant EMP blaster in New Mexico. Don’t worry, it’s here to protect us

An electromagnetic pulse has the potential to disable virtually all electronics within a large area. To help protect against such a threat is a new, friendly EMP emitter. Here's how it works.
Emerging Tech

There’s a new lab-grown meat startup on the block — and it has a secret weapon

Aleph Farms is developing lab-grown steaks with the same flavor, shape, texture, and structure as the real thing using beef cells isolated from living cows. Coming soon to a store near you?
Product Review

This was 3D printed? With the Anycubic Photon, you can't tell

Never mind the fact that the Anycubic Photon 3D printer only costs 500 bucks. In terms of sheer print quality, this printer is on the same level as machines that cost six times as much.
Smart Home

This A.I.-enabled tech brings cutting-edge automation to grocery stores

Takeoff Technologies is working to make grocery deliveries fast, accurate, and convenient using A.I.-enabled technology to augment robotic grocery orders that can be completed in minutes.
Emerging Tech

The best 3D printers of 2018

On the hunt for a new 3D printer? We've got your back. Whether you're a beginner or a seasoned veteran, this list of the best 3D printers has what you're looking for.
Emerging Tech

Postmates’ to roll out Minion-like autonomous delivery robots in 2019

Postmates is about to employ a cute little robot to work alongside its human delivery personnel. Called Serve, the wheel-based bot can carry items weighing up to 50 pounds and has a range of 30 miles.
Emerging Tech

The best drone photos from around the world

Most of today's drones come equipped with high-end cameras, which are quickly revolutionizing the world of aerial photography as we know it. Here are some of the best drone photos from around the world.
Emerging Tech

Are e-cigarettes safe? Here’s what the most recent science says

Ecigarettes are widely regarded and advertised as a healthier alternative to cigarettes for people who are trying to kick the smoking habit. How safe are these cigarette alternatives? We went deep into the recent scientific literature to…
Emerging Tech

Thrill-seekers will be able to pilot themselves in a giant drone as soon as 2019

Want to hitch a ride on a giant drone? The startup Lift Aircraft is gearing up to let paying customers fly its 18-rotor giant drones over assorted scenic landscapes across the U.S.
Emerging Tech

CRISPR gene therapy regulates hunger, staves off severe obesity in mice

Researchers from UC San Francisco have demonstrated how CRISPR gene editing can be used to prevent severe obesity in mice, without making a single edit to the mouse's genome. Here's how.