Skip to main content

Amazon’s facial ID incorrectly identifies members of Congress as criminals

Congress members want a meeting with Jeff Bezos to discuss Rekognition mistakes

ACLU

Amazon may want to go back to the drawing board for this one. A facial recognition tool the retail giant currently offers developers made quite the mistake in a test conducted by the American Civil Liberties Union (ACLU), The New York Times reports. Rather than identifying 28 members of Congress as members of the country’s legislative branch, the tool instead classified these individuals as police suspects. To make matters worse, a disproportionate number of African-American and Latino Congress members were incorrectly identified, raising questions of how far racial profiling has embedded itself even in our machine learning software.

Now, Congress wants answers. Five of the misidentified lawmakers are requesting an “immediate” meeting with Amazon CEO Jeff Bezos. On Thursday, July 26, Reps. Jimmy Gomez, John Lewis, Luis Gutierrez, and Mark DeSaulnier, as well as Sen. Edward Markey, sent letters to the online retail giant asking why they were mismatched. But beyond the personal effects of these cases of mistaken identity, the congressmen are asking for answers as to “how to address the defects of this technology in order to prevent inaccurate outcomes.”

Recommended Videos

Amazon has not indicated yet whether its chief executive will accept these meetings.

In a separate statement, Rep. G.K. Butterfield noted, “I am troubled by the inaccurate outcomes associated with this technology, as there are clear blind spots that will have unintended consequences specifically for people of color. While this technology could have far-reaching economic potential, I encourage Amazon to better train its users on best practices for using this technology, be open and up-front about its limitations, and hire more employees of color who can properly assist with addressing the defects of this technology.”

Among the misidentified lawmakers were Reps. John Lewis and Bobby Rush, both of whom are well-known civil rights leaders and members of the Congressional Black Caucus. But instead of being recognized as elected representatives, these two men (along with 26 others), were falsely flagged as criminals. These mistakes occurred when the ACLU took Amazon’s software to analyze images of all members of Congress against a database of 25,000 publicly accessible mugshots. This resulted in the 28 Congress members being incorrectly identified as lawbreakers, which represents a 5 percent error rate.

ACLU

“This test confirms that facial recognition is flawed, biased, and dangerous,” Jacob Snow, a technology and civil liberties lawyer with the ACLU of Northern California, told The New York Times.

In response to the test, Nina Lindsey, an Amazon Web Services spokeswoman, said that the ACLU used the face matching tool (called Amazon Rekognition) differently from Amazon’s recommendations. “It is worth noting that in real-world scenarios, Amazon Rekognition is almost exclusively used to help narrow the field and allow humans to expeditiously review and consider options using their judgment,” Lindsey said in a statement.

She further pointed out that the ACLU kept the tool’s “confidence threshold” at the default of 80 percent. This means that the group took into account any face matched as 80 percent similar to the police database. However, it’s worth pointing out that Amazon uses the same threshold in an example on its website, showing how employees’ faces can be matched with their ID badges. Lindsey said that in the case of police departments, Amazon recommends that a threshold of 95 percent similarity be implemented to avoid these sorts of mistakes.

Rep. Sanford Bishop (D-Ga.) was falsely identified by Amazon Rekognition as someone who had been arrested for a crime. ACLU

Amazon has previously come under fire for selling Rekognition. In May, 24 civil liberties groups, led by the ACLU, penned a letter to Amazon’s CEO Jeff Bezos requesting that the software cease being sold to law enforcement agencies. The groups expressed concerns that the software could be used to surveil protestors, immigrants, or general members of the public, rather than just police suspects. Amazon employees, investors, and academics have since made similar demands of Amazon.

For the time being, however, that doesn’t seem to be in the cards for the web giant. As Amazon’s spokesperson said, “We remain excited about how image and video analysis can be a driver for good in the world.”

Updated on July 27: Congress is demanding answers from Amazon regarding the mistakes made by its Rekognition software. 

Lulu Chang
Former Digital Trends Contributor
Fascinated by the effects of technology on human interaction, Lulu believes that if her parents can use your new app…
Sen. Josh Hawley calls for criminal antitrust investigation into Amazon
Josh Hawley

Sen. Josh Hawley (R-Missouri) is calling for the Department of Justice to start a criminal antitrust investigation into Amazon. 

Hawley stated in a letter written to Attorney General William Barr on Tuesday that Amazon “has engaged in predatory and exclusionary data practices to build and maintain a monopoly.”

Read more
U.S. military facial recognition could identify people from 1 km away
collage of facial recognition faces

Thanks to recent advances in machine learning, facial-recognition technology can pick faces out of a crowd with impressive accuracy. But just how far away from their subjects can cutting-edge facial-recognition systems work? According to a new report, a whole lot further than you probably think. So far, in fact, that the person identified may not even realize that they were caught on camera in the first place.

New Scientist writes that the United States military is in the process of funding the creation of a portable face-recognition device that's able to identify individuals from up to 1 kilometer away. That’s the equivalent of almost 11 football fields.

Read more
Chinese firm working on facial recognition that can identify you under a mask
Person using cellphone with mask

Let’s say you live in a country where there’s been reports of massive outbreaks of a highly contagious viral infection. Hypothetically, let’s call this infection "coronavirus." To avoid spreading this "coronavirus," many people have taken to wearing medical face masks when going about their everyday life. These masks don’t do much, medically speaking, but it’s considered the polite thing to do.

Now, let’s also say that the government there is notorious for widespread and unregulated use of facial-recognition technology as a way to both fight crime and to identify and silence political dissidents.

Read more