Scientists develop program that matches doodles with photographs

scientists develop program scan doodles match with photographs 5925038217 d38ae770bd b
Ian Norman/Flickr
As FindFace has shown, image recognition has made incredible strides over the past few years. Despite this, artificial intelligence still struggles with certain types of images, such as drawings and doodles. A team of scientists is looking to change that with a new program that could match doodles and sketches to similar photographs.

Scientists believe the solution to bridging this gap is deep learning, a type of artificial intelligence that uses neural networks to analyze and process images. What are neural networks, you ask? Neural networks are brain-like computer systems designed to pick out and create patterns based on inputs.

In an effort to get around the current limitation of image recognition techniques, scientists have started work on a program that will be able to take a doodle or sketch of an image and find a matching photography – a visual search engine, if you will.

One of the main reasons doodles have stumped image recognition programs in the past is that many times, the drawings aren’t perfect representations. Rather than being lifelike recordings of what it is we’re doodling, sketches and drawings are merely how we visualize something.

These less-than-accurate representations, be it on purpose or due to lack of drawing skills, often leads to dramatized proportions and missing elements that might not prevent humans from recognizing the image, but causes problems for computers. For example, that stick figure you drew in third grade likely wouldn’t be recognized as a human by an image recognition program, as its body is nothing more than a few lines placed below a circle – not the lifelike proportions of the human form.

While still in its infancy, New Scientists notes the program has already proven viable. Using more than 600 individuals, scientists displayed a random image for only two seconds and asked the participant to sketch the pictured object from memory. These resulting sketches were put into one neural network while the images shown to the participants were put into another. The two networks then worked together to analyze the sketches and match them to their original picture.

In the initial testing phase, the program was able to match the doodles up to their original image 37 percent of the time. Seemingly unimpressive, it’s important to note that humans only matched the sketches and images correctly 54 percent of the time. While not conclusive, these preliminary numbers show that with a little tweaking and more inputs, the program could very will surpass humans capabilities to match sketches to photos.

Where would this technology be useful though? The possibilities are limitless, really. Looking at it from a consumer standpoint, this tech could be used to create an app that lets you search for images and artwork based on doodles of your own. Imagine being able to draw up a little sketch, scan it with your phone, and find similar artwork to be inspired by or to compare your own work to.

From a more commercial standpoint, one of the most useful features of this technology would be to help police identify criminals by comparing sketch artists renderings to mugshot databases.

More thorough findings and information on the system is set to be released at the SIGGRAPH conference in Anaheim, California, in July.


Uber rolls out rewards program that lets its most loyal riders lock in prices

Uber launched a new loyalty program today called Uber Rewards. It offers frequent riders credits to Uber Eats, car upgrades, and the ability to lock in prices on their most traveled routes.

Alpha Female: Sony awards five women grants to support artisan diversity

Women can face several challenges in launching a photography career -- Sony's latest initiative aims to help propel women in the industry forward. Sony recently announced the winners of the Alpha Female program.

Over a million veterans now eligible for Comcast’s Internet Essentials program

Comcast's low-cost Internet Essentials program, which provides internet access for just $10 per month, has expanded to include U.S. veterans. One million veterans now qualify for the service.

Lyft’s new rewards program promises ride discounts and comfier cars

If you're always hopping in and out of a Lyft car, then you'll be pleased to hear that the ridesharing service is about to launch a rewards program. Perks include discounts on future trips and upgrades to comfier cars.

Golf ball-sized Lume Cube Air is a pocketable LED for photos and video

Off-camera lighting for smartphones and GoPros just got even smaller. Meet the Lume Cube Air, a smaller portable LED light designed for photos and videos that weighs only about two ounces.

Edit portraits with A.I. and adjust focus in the new ON1 Photo RAW 2019 editor

ON1 Photo RAW 2019 now has a dedicated tab for portraits that automatically recognizes faces to help with retouching. The update also brings a new focus stacking tool, enhancements to layers, and improvements to local adjustments.

Taking shots in the dark with Night Sight, the Pixel’s newest photo feature

The Google Pixel range has always been the home of some of the mobile world's best phone cameras. That performance is now getting even better with the introduction of the low-light Night Sight mode.
Emerging Tech

The best drone photos from around the world

Most of today's drones come equipped with high-end cameras, which are quickly revolutionizing the world of aerial photography as we know it. Here are some of the best drone photos from around the world.
Product Review

Fewer pixels, better camera? The Nikon Z6 shows the beauty of restraint

The Nikon Z6 is the sibling to the new mirrorless Z7 -- but for some photographers, the cheaper Z6 may be the better option. Read where the $2,000 camera beats the $3,400 one (and where it doesn’t) in our Nikon Z6 review.
Social Media

Build a wish list and shop videos with Instagram’s latest shopping update

Eyeing a product on Instagram? Now there are more ways to shop from the social network. Instagram just rolled out options to save products in a collection as users can also now shop from videos.

See the National Forests like never before in these awe-inspiring drone videos

What's the difference between a National Park and a National Forest? Drones. With no ban on drones in National Forests -- at least, not yet -- filmmakers have a way to capture the immensity of these locations with stunning results.
Product Review

With outstanding image quality, the ‘basic’ Sony A7 III excels in every way

Replacing the four-year-old A7 II as the new entry-level model in Sony's full-frame line, the A7 III is an impressively capable camera that gives more expensive models a run for their money.
Social Media

Addicted to Instagram? Its new ‘activity dashboard’ is here to help

Ever get that nagging feeling you're spending too much time on Instagram? Well, a new "activity dashboard" has a bunch of features designed to help you better control how you use the addictive photo-sharing app.

Photography news: Best spot for fall photos, new firmware from Fuji and Nikon

Where's the best spot to take fall photos? Michigan, according to social media and a Nikon contest. The results and more in this week's photography news, including significant firmware updates for the Fujifilm X-T3, X-H1, and GFX 50S.