Skip to main content

MIT scientists develop algorithm for removing reflections from photos

Whatever the scenario, we’ve all taken photos through windows hoping to capture that fleeting image: an arching rainbow from the car window, a person or car passing by a building window, or a priceless artifact behind museum glass. And when you go to view your photos afterward, you see that they have been marred by the reflections, which are usually of the photographer. Well, a group at the Massachusetts Institute of Technology may have found a solution to the problem after creating an algorithm that reduces the reflection in photographs.

Photos taken through windows or glass usually have two nearly identical reflections, one slightly offset from the other. Double-paned windows, as well as plain old thick windows, are often the culprits, says YiChang Shih, one of the MIT researchers who developed the algorithm. “With that kind of window, there’s one reflection coming from the inner pane and another reflection from the outer pane,” Shih says. “The inner side will give a reflection, and the outer side will give a reflection as well.”

Recommended Videos

So, the algorithm looks at the values for images inside the glass and outside of it, and attempts to correct it so that it looks the way it was intended. “You have an image from outdoor (A) and another image from indoor (B), and what you capture is the sum of these two pictures (C),” Shih says, who presents the problem as a complex mathematical challenge in then trying to recover A and B data from C.

“But the value of A for one pixel has to be the same as the value of B for a pixel a fixed distance away in a prescribed direction. That constraint drastically reduces the range of solutions that the algorithm has to consider,” MIT News writes.

The process in really technical, but in short, the algorithm uses a technique developed at Hebrew University of Jerusalem in Israel that analyzes the colors of the images by breaking them into 8 x 8 blocks of pixels, calculating “the correlation between each pixel and each of the others.” Out of 197 images from Google and Flickr that they tested the algorithm on, 96 worked. (Check out MIT News for a more detailed explanation.)

“People have worked on methods for eliminating these reflections from photos, but there had been drawbacks in past approaches,” Yoav Schechner, a professor of electrical engineering at Israel’s Technion, told MIT News. “Some methods attempt using a single shot. This is very hard, so prior results had partial success, and there was no automated way of telling if the recovered scene is the one reflected by the window or the one behind the window. This work does a good job on several fronts.”

“The ideas here can progress into routine photography, if the algorithm is further ‘robustified’ and becomes part of toolboxes used in digital photography,” Schechner adds. “It may help robot vision in the presence of confusing glass reflection.”

The MIT researchers will present the algorithm at the Computer Vision and Pattern Recognition conference in Boston, in June. The research is still in its early stages, but over time, we could see it embedded into digital cameras and smartphones, or photo software, and do away with unwanted reflections once and for all.

Enid Burns
Former Contributor
Enid Burns is a freelance writer who has covered consumer electronics, online advertising, mobile, technology electronic…
The new Polaroid Flip comes with sonar autofocus
The Polaroid Flip camera, launched in April 2025.

Polaroid has just unveiled a new camera for some instant analog fun.

The Flip comes with fewer features than Polaroid's pricier I-2 model, but is more advanced than the Go, Polaroid's most basic instant camera -- so it could hit the sweet spot for some folks looking for such a device.

Read more
The best space imagery from Don Pettit’s incredible 7-month mission
The Betsiboka River estuary in Madagascar.

Photographer extraordinaire Don Pettit is about to return to Earth following a seven-month stay aboard the International Space Station (ISS).

During his time in orbit some 250 miles above Earth, Pettit has been sharing breathtaking photos and videos of Earth and beyond, featuring everything from star trails and aurora to rocket launches and cityscapes.

Read more
Best tech deals for mom: A Mother’s Day gift guide
The Garmin Lily 2 smartwatch, in purple and gold colors.

Mother's Day is fast approaching, and for the tech savvy moms and wives out there, we've rounded up the best gift ideas for the special day. With huge discounts up for grabs on these popular devices, it's going to be a good idea to push forward with any purchases as soon as possible -- not only to secure the savings, but also to make sure that your Mother's Day gifts will arrive on time. We've got something for everyone, so read on if you need help finding that perfect gift.

Garmin Lily 2 -- $200 $250 20% off

Read more