MIT scientists develop algorithm for removing reflections from photos

mit scientists develop algorithm for removing reflections from photos reflectionremoval
According to MIT, its “researchers have created a new algorithm that, in a broad range of cases, can automatically remove reflections from digital photos. On the left is the original photo taken through a window, with the photographer's reflection clearly visible. On the right, the reflection has been separated from the photo.” Credit: Massachusetts Institute of Technology
Whatever the scenario, we’ve all taken photos through windows hoping to capture that fleeting image: an arching rainbow from the car window, a person or car passing by a building window, or a priceless artifact behind museum glass. And when you go to view your photos afterward, you see that they have been marred by the reflections, which are usually of the photographer. Well, a group at the Massachusetts Institute of Technology may have found a solution to the problem after creating an algorithm that reduces the reflection in photographs.

Photos taken through windows or glass usually have two nearly identical reflections, one slightly offset from the other. Double-paned windows, as well as plain old thick windows, are often the culprits, says YiChang Shih, one of the MIT researchers who developed the algorithm. “With that kind of window, there’s one reflection coming from the inner pane and another reflection from the outer pane,” Shih says. “The inner side will give a reflection, and the outer side will give a reflection as well.”

So, the algorithm looks at the values for images inside the glass and outside of it, and attempts to correct it so that it looks the way it was intended. “You have an image from outdoor (A) and another image from indoor (B), and what you capture is the sum of these two pictures (C),” Shih says, who presents the problem as a complex mathematical challenge in then trying to recover A and B data from C.

“But the value of A for one pixel has to be the same as the value of B for a pixel a fixed distance away in a prescribed direction. That constraint drastically reduces the range of solutions that the algorithm has to consider,” MIT News writes.

The process in really technical, but in short, the algorithm uses a technique developed at Hebrew University of Jerusalem in Israel that analyzes the colors of the images by breaking them into 8 x 8 blocks of pixels, calculating “the correlation between each pixel and each of the others.” Out of 197 images from Google and Flickr that they tested the algorithm on, 96 worked. (Check out MIT News for a more detailed explanation.)

“People have worked on methods for eliminating these reflections from photos, but there had been drawbacks in past approaches,” Yoav Schechner, a professor of electrical engineering at Israel’s Technion, told MIT News. “Some methods attempt using a single shot. This is very hard, so prior results had partial success, and there was no automated way of telling if the recovered scene is the one reflected by the window or the one behind the window. This work does a good job on several fronts.”

“The ideas here can progress into routine photography, if the algorithm is further ‘robustified’ and becomes part of toolboxes used in digital photography,” Schechner adds. “It may help robot vision in the presence of confusing glass reflection.”

The MIT researchers will present the algorithm at the Computer Vision and Pattern Recognition conference in Boston, in June. The research is still in its early stages, but over time, we could see it embedded into digital cameras and smartphones, or photo software, and do away with unwanted reflections once and for all.

Photography

Not just for Lightroom anymore, Loupedeck+ now works with Photoshop

Loupedeck+ can now help photographers edit in Photoshop too, thanks to physical controls for swapping tools, running actions, and more. The photo-editing console expanded to include Photoshop in the list of compatible editing programs.
Emerging Tech

The best drone photos from around the world

Most of today's drones come equipped with high-end cameras, which are quickly revolutionizing the world of aerial photography as we know it. Here are some of the best drone photos from around the world.
Deals

Want a fun, affordable instant camera? The Fujifilm Instax Mini 7S is just $49

Instant cameras have had a surprising resurgence of late, and no brand is better recognized in the instant photo space today than Fujifilm Instax. Walmart is currently offering the Instax Mini 7S for just $49.
Mobile

New Moment photo cases and lenses are a big plus for the OnePlus 6T

Moment has announced that it is now releasing photo cases and lenses for the OnePlus 6 and OnePlus 6T, giving owners of those phones the ability to add on some high-quality photographic accessories.
Photography

Photographers can now customize the layout of Lightroom Classic controls

Tired of scrolling past Lightroom tools that you don't use? Adobe Lightroom Classic now allows users to reorganize the Develop panel. The update comes along with new sharing options in Lightroom CC, and updates to the mobile Lightroom app.
Social Media

Instagram could be making a special type of account for influencers

Instagram influencers fall somewhere between a business profile and a typical Instagram, so the company is working on developing a type of account just for creators. The new account type would give creators more access to analytical data.
Mobile

Leave the laptop at home, the iPad Pro is the travel buddy to take on vacay

The iPad Pro is a powerful tablet that's perfect for creatives and professionals. How does it fare when traveling with it as a laptop replacement? We took it on a two week trek in Japan to find out.
Cars

Best Products of 2018

Our reception desk has so many brown boxes stacked up, it looks like a loading dock. We’re on a first-name basis with the UPS guy. We get new dishwashers more frequently than most people get new shoes. What we’re trying to say is: We…
Photography

These are the best action cameras money can buy

Action cameras are great tools for capturing videos of your everyday activities, whether it's a birthday party or the steepest slope you've ever descended on your snowboard. These are the best money can buy.
Photography

Canon holiday sale features the Rebel T6 2-lens kit for just $449

If you have a budding photographer in your life in need of a real camera, the Canon EOS Rebel T6 could make the perfect gift. Canon is currently offering the camera in a two-lens bundle for just $449 through December 29.
Emerging Tech

Light, speed: Lighting kit for DJI Mavic 2 lets you fly and film in the dark

Lume Cube, maker of small battery-powered LED lights for mobile photography, has announced a new lighting kit built specifically for the DJI Mavic 2 -- the first of its kind. Already our favorite drone, this makes the Mavic 2 even better.
Social Media

Instagram’s 2018 year in review shines a light on where our hearts are

What did Instagram users share the most in 2018? A lot of heart emojis, heart face filters, and heart GIFs. The platform recently shared the year's top trends, including hashtags like #fortnite and #metoo along with a few surprises.
Computing

Need a monitor for professional photo-editing? These are the very best

Looking for the best monitor for photo editing? You'll need to factor in brightness, color accuracy, color gamut support and more. Fortunately, we've rounded up the best ones for you, to help you make an educated purchase.
Photography

2018 was a terrific year for new cameras. Here are our favorites

This year, we saw the unveiling of several exciting new cameras. We look back at our favorite models -- each delivering something unique, yet all are powerful and feature-rich. Camera fans, rejoice!