Can an algorithm be racist? Spotting systemic oppression in the age of Google

Can a bridge be racist?

It sounds ridiculous, but that’s exactly the argument sociologist Langdon Winner makes in “Do Artifacts Have Politics?” — a classic essay in which he examines several bridges built over roadways in Long Island, New York.

Many of these bridges were extremely low, with just nine feet of clearance from the curb. Most people would be unlikely to attach any special meaning to their design, yet Winner suggested that they were actually an embodiment of the social and racist prejudices of designer Robert Moses, a man who was responsible for building many of the roads, parks, bridges, and other public works in New York between the 1920s and 1970s.

algorithms of oppression racist

With the low bridges, Winner wrote that Moses’ intention was to allow only whites of “upper” and “comfortable middle” classes access to the public park, since these were largely the only demographics able to afford cars at the time. Because poorer individuals (which included many people of color) relied on taller public buses, they were denied access to the park, as these buses were unable to handle the low overpasses and were therefore forced to find alternative routes.

As New York town planner Lee Koppleman later recalled, “The old son of a gun … made sure that buses would never be able to use his goddamned parkways.”

Jump forward the best part of 40 years and Dr. Safiya Umoja Noble, part of the faculty at the University of Southern California (USC) Annenberg School of Communication, has written a book that updates Langdon Winner’s critique for the digital age.

Noble’s Algorithms of Oppression makes the argument that many of the algorithms driving today’s digital revolution (she focuses particularly on those created by Google) are helping to marginalize minorities through the way that they structure and encode the world around us. They are, quite literally, a part of systemic racism.

Discriminatory patterns

Before she earned her PhD, Noble was in the advertising industry where, she told Digital Trends, one of the big discussions was about “how to game Google for our clients, because we knew that if we could get content about our clients on the first page, that’s what mattered.”

A few years later, she glimpsed this world of search engine optimization and prioritization from another angle when a friend mentioned the search results presented when a person looks for the term “black girls.”

algorithms of oppression racist safiya noble
Dr. Safiya U. Noble, Author of Algorithms of Oppression

“The first page was almost exclusively pornography or highly sexualized content,” she said. “I thought maybe it was a fluke, but over the next year I did the same for other identities, such as Asian girls and Latinas.”

The same thing held true: frequent pornographic results, even when the search terms didn’t include suffixes like “sex” or “porn.” “That’s when I started taking seriously that this wasn’t just happening in a random way, and thought that it was time for a more systemized study.”

“…I started taking seriously that this wasn’t just happening in a random way.”

Noble isn’t the first person to spot worrying discrimination embedded into tools that many of us still believe are objective. Several years ago, the African-American Harvard University PhD Latanya Sweeney noticed that her search results were accompanied by ads asking, “Have you ever been arrested?” These ads did not appear for her white colleagues.

Sweeney began a study ultimately demonstrating that the machine-learning tools behind Google’s search were inadvertently racist, linking names more commonly given to black people to ads relating to arrest records.

It’s not just racial discrimination, either. Google Play’s automated recommender system has been found to suggest that those who download Grindr, a location-based social-networking tool for gay men, also download a sex offender location-tracking app.

In both cases, the issue wasn’t necessarily that there was a racist programmer responsible for the algorithm, but rather that the algorithms were picking up on frequent discriminatory cultural associations between black people and criminal behavior and homosexuality and predatory behavior.

Who has the responsibility?

Noble makes the point in her book that companies like Google are now so influential that they can help shape public attitudes, as well as reflect them.

“We are increasingly being acculturated to the notion that digital technologies, particularly search engines, can give us better information than other human beings can,” she said.

“The idea is that they are vetting the most important information, and provide us with [objective answers] better than other knowledge spheres. People will often take complex questions to the web and do a Google search rather than going to the library or taking a class on the subject. The idea is that an answer can be found in 0.3 seconds to questions that have been debated for thousands of years.”

To Noble, the answer is that tech giants need to be held accountable for the results that they provide — and the harm they might cause.

“If your platform allows this kind of content to flourish, then you’re also responsible.” 

“Tech companies have really invested in lobbying in the U.S. that they are simply intermediaries,” she continued. “They [claim that they] are tech companies and not media companies; they’ve designed a platform but they are not responsible for the content that flows through it. In the U.S. they do that because it means they are held to be harmless for trafficking in [things like] anti-semitism, Nazi propaganda, white supremacist literature, child pornography, and all the most hideous dimensions of the things which are out there on the web.”

She has little patience for the suggestion that tech companies like Google are simply reflecting what users search for. This is an argument Google itself made several years ago when it was taken to court in Germany for allegedly defamatory autocomplete results, linking the name Bettina Wulff, wife of the former German president Christian Wulff, with a rumor about prostitution and escorts.

“We believe that Google should not be held liable for terms that appear in autocomplete as these are predicted by computer algorithms based on searches from previous users, not by Google itself,” Google said at the time.

“If your platform has been designed in such a way as to allow this kind of content to flourish, then you’re also responsible for the way that you’ve designed your platform,” Noble said. “You cannot absolve your company from that.”

Asking the right questions

She also argues that tech companies are not as far removed from other, more traditional companies as they might like.

“When we’re talking about corporations and their impact on society,” she said. “I don’t think there are many industries that we can trace specific actions to one individual. What typically happens is that a CEO or a board of directors are held accountable if there is harm that hits communities. The tech industry doesn’t have to be different; it’s not really that different from fossil fuels or other industries that might cause harm through a particular product that’s been developed.”

algorithms of oppression racist opp feat

Finally, she calls for better training of those developing the algorithms that dictate which information is shown to us.

“If you are going to design technology for society, you should have a deep education on societal issues.”

“One of the things that is particularly frightening to me is that many of the people who are designing these technologies, and embedding their own values and world views and best judgment into them, have very limited education around the liberal arts, humanities, or the social sciences,” Noble said. “They typically come into engineering curriculums where they are hyper-focused on theoretical or applied math … A lot of them are operating on twelfth-grade level humanities instruction … If you are going to design technology for society, you should have a deep education on societal issues.”

The arguments she makes in Algorithms of Oppression are incisive and provocative. While Google and other tech companies have taken steps to solve some of the most egregiously high profile issues (my top search for “black girls” leads to Black Girls Code, a San Francisco tech training initiative for underrepresented youth), problems such as this will happen with greater frequency as platforms such as Google, Facebook, and others play an increasing role in our lives, with ever more information to parse.

There are no easy solutions. How much do search results influence public opinion? Are tech companies qualified to make value judgements about what is and isn’t acceptable? Should search engines display answers that are purely based on what gets clicks, even when these results are offensive or even harmful? Will making companies responsible for their content mean that they err on the side of censorship, even when the material doesn’t warrant it? These are conundrums that companies like Google will face as they grow ever larger.

Dr. Safiya Umoja Noble doesn’t have all the answers. But she’s asking the right questions.

Home Theater

Looking for a podcast? Pandora’s Podcast Genome Project is just the ticket

Pandora has created a brand new service for podcast listeners. Powered by calls the Podcast Genome Project, the new podcast offering uses an advanced combination of algorithms and human curation to deliver the perfect podcasts for you.
Movies & TV

The best shows on Netflix, from 'The Haunting of Hill House’ to ‘The Good Place’

Looking for a new show to binge? Lucky for you, we've curated a list of the best shows on Netflix, whether you're a fan of outlandish anime, dramatic period pieces, or shows that leave you questioning what lies beyond.
Home Theater

The best movies on Netflix in November, from 'The Witch’ to ‘Dracula’

Save yourself from hours wasted scrolling through Netflix's massive library by checking out our picks for the streamer's best movies available right now, whether you're into explosive action, witty humor, or anything else.
Movies & TV

The 10 best 'Black Mirror' episodes are thought-provoking, disturbing, mesmerizing

After watching all 19 episodes across four seasons and one special, we've selected and ranked the best Black Mirror episodes released so far. Read on to find out if your favorite episode from the award-winning Netflix series made the list.
Emerging Tech

‘Bionic mushroom’ can generate electricity without using fossil fuels

Researchers have come up with a way to produce electricity without fossil fuels using mushrooms covered with bacteria. The mushroom provides a safe environment for special cyanobacteria that generate electricity when light is shone on them.
Emerging Tech

Curiosity rover active and drilling again after computer issue

The Curiosity rover has succeeded in drilling a hole into the tough bedrock that previously defeated it, allowing imaging and collection of samples. The rover had been incapacitated for a few weeks due to problems with its computer.
Emerging Tech

Awesome Tech You Can’t Buy Yet: 1-handed drone control, a pot that stirs itself

Check out our roundup of the best new crowdfunding projects and product announcements that hit the web this week. You may not be able to buy this stuff yet, but it sure is fun to gawk!
Emerging Tech

Astronomers discover two rogue planets that do not orbit a star

Astronomers have identified two rogue planets in our galaxy which do not orbit around a star. Unlike the vast majority of discovered planets, these rogue planets drift through space alone with no sun to shine on them.
Emerging Tech

Pairs of supermassive black holes spotted in colliding galaxies

Astronomers have discovered several pairs of supermassive black holes in galaxies that are colliding with each other. These black holes will spiral closer and closer together and eventually merge into one supermassive black hole.
Emerging Tech

Quantum-based accelerometer can locate objects without GPS

Researchers have created a quantum "compass" that allows navigation without satellites. The instrument, technically called a standalone quantum accelerometer, is small enough to be transportable and has a very high level of accuracy.
Emerging Tech

Ancient continent discovered beneath the ice of Antarctica

Antarctica could be hiding the remains of a long-lost continent. Scientists created a 3D map of the crust beneath the Antarctic ice sheet which shows a similarity to the crust in Australia and India, suggesting they used to be joined.
Emerging Tech

Rocket Lab steps into spotlight with its first commercial rocket launch

Rocket Lab has deployed multiple small satellites into orbit in its first notable commercial launch. Its New Zealand-born boss said the success means "rapid and reliable access to space is now a reality for small satellites."
Emerging Tech

Alibaba’s Singles’ Day sale smashes online shopping records

The annual online shopping frenzy that is Singles' Day this year raked in $30.8 billion, up from $25 billion last time around. The Alibaba-organized event generates more in sales than Black Friday and Cyber Monday combined.
Emerging Tech

Watch this lab-grown heart tissue beat just like the real thing

A team of researchers in Germany have used stem cells to create a lab-grown human heart tissue which actually beats, as well as responding to drugs in the same way as the real thing.