Skip to main content

We’re closer to China’s disturbing ‘Social Credit System’ than you realize

Have you ever heard the saying that you are the average of the five people that you spend the most time with? It’s quite possible that someone in the Chinese government is intimately acquainted with the idea. Over the past several years, the country has been working to quantify the trustworthiness of every citizen in China. A bit like a financial credit score, only applying to how much stock we can put in a person’s character, the idea is that using big data and A.I. algorithms to analyze trustworthiness can lead to a new era in upstanding citizenry.

Or as the original proposal for the system, titled “Planning Outline for the Construction of a Social Credit System,” put it: “[Such an initiative] will forge a public opinion environment where keeping trust is glorious. It will strengthen sincerity in government affairs, commercial sincerity, social sincerity and the construction of judicial credibility.”

So where does businessman and motivational speaker Jim Rohn’s quote about a person being the sum total of five other people come into it? Because one element of the Social Credit System — which will become mandatory for citizens by 2020 — depends on who you associate with online.

It’s an idea straight out of the oft-invoked George Orwell dystopia Nineteen Eighty-Four.

In addition to more mundane areas like whether you pay your community charge on time, the system’s reputational algorithm will also factor in your choice of online friends. That person who complains about how the government is doing its job could suddenly cost you some serious social cred. Befriend too many wrongthinkers and you could quickly find yourself classed as a wrongthinker too.

It’s an idea straight out of the oft-invoked George Orwell dystopia Nineteen Eighty-Four. But it’s also not wholly unique to China. True, the U.S. government isn’t publicly instituting a Social Credit System, but the idea that digital reputation analysis isn’t something that affects us all in 2018 is patently untrue.

Your social standing, algorithmized

The recent ongoing Facebook saga involving potential user data abuse underlines once again the ways in which our online personas are curated, collected, monetized and sliced and diced in all sorts of ways. Increasingly they will be treated as next-generation credit scores, not only able to keep tabs on our financial respectability but also manner of other metrics by which we can be considered quality citizens.

Recently, I failed such a test. On Twitter, I try to follow a diverse range of thinkers, since it seems a good way to steer clear of the “filter bubble” effect. If there’s an issue that I feel strongly about, I like to follow “thought leaders” on the other side of the ideological divide so as to be exposed to their response to items in the news.

Westend61/Getty Images

Who knows: a well-argued case might even change your mind about something you thought was true. At the very least, it gives you a chance to know the most compelling arguments used against your position. All was going well until I discovered that I had been blocked by a certain user I’ll call “user X” with whom I had had no previous interaction.

Based on a bit of further digging I discovered the explanation: another person I followed had been added to an automated blocklist by user X, so that anyone who followed them was then barred from viewing the tweets of user X. Somewhere in the ephemeral world of the interwebs I felt my personal social credit noticeably slide in a way that it hadn’t since my middle school classmates discovered I still watched Power Rangers.

On the internet, everyone knows you’re a dog

We’re still getting to grips with the way that our digital identities are changing. Twenty-five years ago, we didn’t worry about online reputation because the entire galvanizing idea of the internet was built around anonymity. A famous 1993 New Yorker cartoon depicted two computer-savvy canines using a computer. “On the Internet, nobody knows you’re a dog,” the tagline read. Today, not only do companies online know that you’re a dog; they know which brand of dog food you enjoy, what collar you wear, where you stretch your legs and — for better or worse — where you do your messes.

Igniyte UK - Why search engine results matter for your online reputation

Here in 2018, we take it for granted that the internet knows who we are. It’s considered an insult if it doesn’t. If websites don’t know who we are then we don’t get personalized news recommendations, social media friend updates, or suggested videos. Heck, on Twitter, the idea of being “verified” as yourself is just about the ultimate reward you can have: an idea that would have struck early web-users as bizarre.

Studies have suggested that social media profiles can also be used to accurately predict our employment success.

While we may have multiple personas online, just as we do in the real world, these identities aren’t neatly self contained. They can spill over into other areas of our lives; the line increasingly blurred between public and private personas, or social and professional ones. For instance, multiple studies have suggested that social media profiles (designed for, you know, our social lives) can also be used to accurately predict our employment success. One 2016 research project carried out by researchers at Belgium’s Ghent University came to the conclusion that Facebook profile pictures can increase or decrease your chances of being offered an interview for a job by up to 40 percent.

“I think the CV of now and the future moves from a paper one we send out to how we look when Googled,” Simon Wadsworth, a managing partner Igniyte, an online reputation management company, told Digital Trends. “We know that’s the first port of call for employers. Looking your best online will be a major factor when anyone is looking to go into higher education, get a job, or even buy a house.”

A game you have to play

Chris DeGraw/Digital Trends

Don’t think you can get away without playing, either. “I think also there is an issue if a person can’t be found online in any way – so a happy medium is required,” he continued. Not being discoverable online could mean that a person has something to hide. More mundanely, but crucially, it might also simply leave a person out in the digital cold. “For the generation going to college and getting jobs, it suggests not being very digitally savvy if they have no presence online,” Wadsworth said.

If a good Facebook profile makes your more likely to get an interview, what does no Facebook profile at all mean?

The problematic nature of digital identities will only become more complex as we continue to live increasingly online. Credit scores measure one aspect of our reputation: our ability to pay back money. They take into account things like whether payments are made on time, how much of our line of credit we’ve used, the length of our credit history, types of credit we use, and our past credit applications. From there, a FICO score is created and used by banks and retailers.

The idea behind the FICO score dates back to the 1950s, before it was ultimately introduced in 1989. Today, there is a wealth of information that can be gathered about users that could not possibly be known about in the 1980s. What a person “likes” online, what they buy, where they buy it, who they socialize with, what they do in their homes, all of it can increasingly be gathered and combed through using smart machine learning tools.

As more and more of ourselves are virtualized, what is possible to “know” expands. Fitness data, health records, online dating profiles and preferences, our emotional states, how we learn in the classroom, how quickly we read e-books on our vacation and much more can build up an astonishingly accurate picture of who we are. True, much of this data is anonymized, but the ability to draw on this information and cross-reference it in all sorts of new ways is an unavoidable reality.

The return of social credit

The system that credit scores replaced was based on reputation and qualitative judgements. A person could be denied a line of credit because a banker didn’t like their demeanor. In addition to payment history, this meant that judgements were the result of things like home visits, a person’s perceived standing in the community and more. Credit scores were intended to take the subjectivity and, just as importantly, opacity out of this process.

Today, social credit is once again a reality.

Today, social credit is once again a reality. Everything from the information that we are shown to the prices and opportunities we’re offered are wrapped up in the byzantine way the digital world curates identity. We have easy access to our credit scores, and know the way to improve them. When it comes to the online world, this isn’t nearly so straightforward. Who is gathering data about us, how is it used and what does this mean for the way that we perceive the digital world around us; all of these questions are far from clear.

Projects like China’s Social Credit System should worry people for a number of reasons. But just like the sci-fi dystopias it’s being likened to, it scares us because we innately understand that it’s a world we’re familiar with — only with the volume cranked up to eleven. Don’t for a second think that the impetus driving it isn’t the same one finding its way into every interaction that we have online.

Editors' Recommendations

Luke Dormehl
I'm a UK-based tech writer covering Cool Tech at Digital Trends. I've also written for Fast Company, Wired, the Guardian…
This AI cloned my voice using just three minutes of audio
acapela group voice cloning ad

There's a scene in Mission Impossible 3 that you might recall. In it, our hero Ethan Hunt (Tom Cruise) tackles the movie's villain, holds him at gunpoint, and forces him to read a bizarre series of sentences aloud.

"The pleasure of Busby's company is what I most enjoy," he reluctantly reads. "He put a tack on Miss Yancy's chair, and she called him a horrible boy. At the end of the month, he was flinging two kittens across the width of the room ..."

Read more
Digital Trends’ Top Tech of CES 2023 Awards
Best of CES 2023 Awards Our Top Tech from the Show Feature

Let there be no doubt: CES isn’t just alive in 2023; it’s thriving. Take one glance at the taxi gridlock outside the Las Vegas Convention Center and it’s evident that two quiet COVID years didn’t kill the world’s desire for an overcrowded in-person tech extravaganza -- they just built up a ravenous demand.

From VR to AI, eVTOLs and QD-OLED, the acronyms were flying and fresh technologies populated every corner of the show floor, and even the parking lot. So naturally, we poked, prodded, and tried on everything we could. They weren’t all revolutionary. But they didn’t have to be. We’ve watched enough waves of “game-changing” technologies that never quite arrive to know that sometimes it’s the little tweaks that really count.

Read more
Digital Trends’ Tech For Change CES 2023 Awards
Digital Trends CES 2023 Tech For Change Award Winners Feature

CES is more than just a neon-drenched show-and-tell session for the world’s biggest tech manufacturers. More and more, it’s also a place where companies showcase innovations that could truly make the world a better place — and at CES 2023, this type of tech was on full display. We saw everything from accessibility-minded PS5 controllers to pedal-powered smart desks. But of all the amazing innovations on display this year, these three impressed us the most:

Samsung's Relumino Mode
Across the globe, roughly 300 million people suffer from moderate to severe vision loss, and generally speaking, most TVs don’t take that into account. So in an effort to make television more accessible and enjoyable for those millions of people suffering from impaired vision, Samsung is adding a new picture mode to many of its new TVs.
[CES 2023] Relumino Mode: Innovation for every need | Samsung
Relumino Mode, as it’s called, works by adding a bunch of different visual filters to the picture simultaneously. Outlines of people and objects on screen are highlighted, the contrast and brightness of the overall picture are cranked up, and extra sharpness is applied to everything. The resulting video would likely look strange to people with normal vision, but for folks with low vision, it should look clearer and closer to "normal" than it otherwise would.
Excitingly, since Relumino Mode is ultimately just a clever software trick, this technology could theoretically be pushed out via a software update and installed on millions of existing Samsung TVs -- not just new and recently purchased ones.

Read more