Skip to main content

Image-recognition A.I. has a big weakness. This could be the solution

You’re probably familiar with deepfakes, the digitally altered “synthetic media” that’s capable of fooling people into seeing or hearing things that never actually happened. Adversarial examples are like deepfakes for image-recognition A.I. systems — and while they don’t look even slightly strange to us, they’re capable of befuddling the heck out of machines.

Several years ago, researchers at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (CSAIL) found that they could fool even sophisticated image recognition algorithms into confusing objects simply by slightly altering their surface texture. These weren’t minor mix-ups, either.

Related Videos
Image Recognition Turtle Recognized as a Rifle

In the researchers’ demonstration, they showed that it was possible to get a cutting-edge neural network to look at a 3D-printed turtle and see a rifle instead. Or to gaze upon a baseball and come away with the conclusion that it is an espresso. Were such visual agnosia to manifest in a human, it would be the kind of neurological case study that would find its way into a book like Oliver Sacks’ classic The Man Who Mistook His Wife for a Hat.

Adversarial examples represent a fascinating vulnerability when it comes to how visual A.I. systems view the world. But they also, as you might expect from a flaw that confuses a novelty toy turtle with a rifle, represent a potentially alarming one. It’s one that researchers have been desperately figuring out how to patch.

Now, another group of researchers from MIT have come come up with a new system that could help to dodge “adversarial” inputs. In the process, they have imagined a frankly terrifying use case for adversarial examples, one that could, if implemented by hackers, be used to deadly effect.

The scenario is this: Autonomous cars are getting better and better at perceiving the world around them. But what if, suddenly, the visual input-based onboard cameras in a car were either purposely or accidentally rendered unable to identify what was in front of them? Miscategorizing an object on the road — such as failing to correctly identify and place a pedestrian — could potentially end very, very badly indeed.

Fending off adversarial attacks

“Our group has been working at the interface of deep learning, robotics, and control theory for several years — including work on using deep RL [reinforcement learning] to train robots to navigate in a socially aware manner around pedestrians,” Michael Everett, a postdoctoral researcher in the ‎MIT Department of Aeronautics and Astronautics, told Digital Trends. “As we were thinking about how to bring those ideas onto bigger and faster vehicles, the safety and robustness questions became the biggest challenge. We saw a great opportunity to study this problem in deep learning from the perspective of robust control and robust optimization.”

Socially Aware Motion Planning with Deep Reinforcement Learning

Reinforcement learning is a trial-and-error-based approach to machine learning that, famously, has been used by researchers to get computers to learn to play video games without being explicitly taught how. The team’s new reinforcement learning and deep neural network-based algorithm is called CARRL, short for Certified Adversarial Robustness for Deep Reinforcement Learning. In essence, it’s a neural network with an added dose of skepticism when it comes to what it’s seeing.

In one demonstration of their work, which was supported by the Ford Motor Company, the researchers built a reinforcement learning algorithm able to play the classic Atari game Pong. But, unlike previous RL game players, in their version, they applied an adversarial attack that threw off the A.I. agent’s assessment of the game’s ball position, making it think that it was a few pixels lower than it actually was. Normally, this would put the A.I. player at a major disadvantage, causing it to lose repeatedly to the computer opponent. In this case, however, the RL agent thinks about all the places the ball could be, and then places the paddle someplace where it won’t miss regardless of the shift in position.

“This new category of robust deep learning algorithms will be essential to bring promising A.I. techniques into the real world.”

Of course, games are vastly more simplified than the real world, as Everett readily admits.

“The real world has much more uncertainty than video games, from imperfect sensors or adversarial attacks, which can be enough to trick deep learning systems to make dangerous decisions — [such as] spray-painting a dot on the road [which may cause a self-driving car] to swerve into another lane,” he explained. “Our work presents a deep RL algorithm that is certifiably robust to imperfect measurements. The key innovation is that, rather than blindly trusting its measurements, as is done today, our algorithm thinks through all possible measurements that could have been made, and makes a decision that considers the worst-case outcome.”

In another demonstration, they showed that the algorithm can, in a simulated driving context, avoid collisions even when its sensors are being attacked by an adversary that wants the agent to collide. “This new category of robust deep learning algorithms will be essential to bring promising A.I. techniques into the real world,” Everett said.

More work still to be done

It’s still early days for this work, and there’s more that needs to be done. There’s also the potential issue that this could, in some scenarios, cause the A.I. agent to behave too conservatively, thereby making it less efficient. Nonetheless, it’s a valuable piece of research that could have profound impacts going forward.

“[There are other research projects] that focus on protecting against [certain types] of adversarial example, where the neural network’s job is to classify an image and it’s either right [or] wrong, and the story ends there,” Everett said, when asked about the classic turtle-versus-rifle problem. “Our work builds on some of those ideas, but is focused on reinforcement learning, where the agent has to take actions and gets some reward if it does well. So we are looking at a longer-term question of ‘If I say this is a turtle, what are the future implications of that decision?’ and that’s where our algorithm can really help. Our algorithm would think about the worst-case future implications of choosing either a turtle or a rifle, which could be an important step toward solving important security issues when A.I. agents’ decisions have a long-term effect.”

A paper describing the research is available to read on the electronic preprint repository arXiv.

Editors' Recommendations

Here’s what a trend-analyzing A.I. thinks will be the next big thing in tech
brain network on veins illustration

Virtual and augmented reality. 3D printing. Natural language processing. Deep learning. The smart home. Driverless vehicles. Biometric technology. Genetically modified organisms. Brain-computer interfaces.

These, in descending order, are the top 10 most-invested-in emerging technologies in the United States, as ranked by number of deals. If you want to get a sense of which technologies will be shaping our future in the years to come, this probably isn’t a bad starting point.

Read more
Read the eerily beautiful ‘synthetic scripture’ of an A.I. that thinks it’s God
ai religion bot gpt 2 art 4

Travis DeShazo is, to paraphrase Cake’s 2001 song “Comfort Eagle,” building a religion. He is building it bigger. He is increasing the parameters. And adding more data.

The results are fairly convincing, too, at least as far as synthetic scripture (his words) goes. “Not a god of the void or of chaos, but a god of wisdom,” reads one message, posted on the @gods_txt Twitter feed for GPT-2 Religion A.I. “This is the knowledge of divinity that I, the Supreme Being, impart to you. When a man learns this, he attains what the rest of mankind has not, and becomes a true god. Obedience to Me! Obey!”

Read more
Cailifornia has a blackout problem. Could giant flow batteries be the solution?
flow batteries california powerlines

When it comes to electricity, California has it rougher than most states. It's home to more electricity-consuming residents than anywhere else in the country, and when demand for electricity spikes (often during summer heat waves), the state is sometimes unable to supply all of those residents with power. For this reason, California’s energy authorities will occasionally implement planned power outages known as rolling blackouts. If you've ever been caught in 100-degree heat with no air conditioning, you know that's less than ideal.

One of the ways California is planning on dealing with this problem is by setting up large battery grids that state officials hope will keep things running when extreme weather hits. These batteries can be charged by the state’s growing network of solar panels and wind turbines, and they can kick in when there’s an energy shortage to supply power.

Read more