AI may not be the boon to authoritarian regimes and dictatorships that they expect.

China is in the news right now for a lot of reasons. Trade wars, viral outbreaks, the repression of millions of Muslim Uighurs, economic ascension, Hong Kong and the militarization of the South China Sea are only just a few of the headline stories. Each topic has received tens to hundreds millions of opinions from both inside and outside the country, with thinkers and voices everywhere pointing to structural trends and underlying shifts that will cause the world to tip ever more to Asia in the coming century. One area of study popular in the West is criticism of the state’s economic and industrial model. Pundits have made careers off of identifying when the country’s state-owned banks might collapse, how their energy mix is likely to impact climate change and why their economic model will eventually face its reckoning. There are multitudes of differing opinions on each side. However, there is one view that is held universally by virtually all pundits: China’s development of a surveillance state is a godsend to its stability-obsessed rulers and a major risk for the rest of the world.

Anyone who owns a smartphone, lives in a city or goes on the internet can be tracked or listened to at virtually any time. But in Western democracies, most governments don’t monitor their citizens. Specific targets may be identified and tracked or individuals placed on watchlists, but broad monitoring of an entire population is not undertaken systematically. Equally importantly is that even were this activity to take place, constitutional rights to freedom of speech and expression protect individuals who are not directly breaking the law from being penalized for expressing their opinions. This limits the inevitability and consequences of surveillance and ensures that citizens have legal protections against unlawful government action that can be used to hold decision-makers accountable. In the case of an authoritarian state, none of that is true. In fact, surveillance capabilities are usually viewed as a net positive as they allow for a regime to monitor their citizens, facilitates social control and makes it easier to target dissidents. Expression of opinion that isn’t in line with that of the ruling party is often punished, meaning that this surveillance has real consequences. Curiously, this action is not always done with citizen opposition – in fact, most of the time, citizens are entirely aware of these activities and may even condone their actions.

Understanding why this view on the value of encroached freedom is held requires a step back from the case of China towards a broader discussion of how authoritarian states stay in power. Citizens support authoritarian governments when they believe that the stability, direction or national benefits they offer outweigh the sacrifices of freedom of speech and action. This line – the divide between tolerable and intolerable repression – is the relationship between governments and citizens in all states, but the perception of the value of stability and the risks of its disruption are especially prevalent in dictatorships. Most citizens under authoritarians are willing to accept high levels of government monitoring and the repression of minorities for three major reasons: one, the government has grown the economy fast enough that the country’s citizens have directly benefitted from its leadership. This creates ” output legitimacy” (i.e. the view that the state knows best because the state has been successful) in the eyes of voters that the government is acting with knowledge and purpose. It also creates a tenuous balancing act. In democracies, when the economy shrinks or an emergency is poorly handled, voters can hold government accountable by voting them out of office (i.e. democracies allow “steam out of the system” by easing political tensions through elections). For dictators, there is no mechanism for releasing steam except to diffuse tension (by force) or watch it explode (by being forcibly removed from office). That means for authoritarians to continue to be accepted by their people, they need to continuously prove that they should be in charge by making peoples lives “better” almost constantly. As soon as they can’t, they must either find someone to blame for it (which only works if people believe them), directly repress their people with the help of the military or risk being swiftly and forcefully removed from office.

The second reason why people support authoritarians is because the value of the political stability that comes with dictatorship is often overstated by propagandists. The government says they need to take major steps to conserve safety and stability, such as monitoring or censorship of information. People tend to believe them, which is helped by the fact that authoritarians often control the flow of information to the public. This means that national narratives can be crafted that justify any measure of actions taken by governments including acts of war, repressions of minorities or removal of rights. This creates a feedback loop: People believe the government knows best because the economy is growing, and they therefore believe the government knows what they’re doing when they justify taking steps to oppress people or remove rights. This allows the government to take ever more drastic steps and more greatly control the flow of information. The third reason people support dictators is, simply put, they’re scared not to. This speaks to the view that dictators offer social stability and that stability requires people to fall into place with the authoritarian’s view on what society should be, lest their alternative views land them somewhere dark.

These three reason help shape an understanding about why Chinese nationals broadly accept China’s use of AI to develop a surveillance state. The nation’s camera network (which forms the largest video surveillance system in the world), internet censorship and currently under development “social credit system” terrify most of the world, but are seen in China as being important to ensure safety. If people are constantly being monitored, it’s seen as reducing incidence of crime; one cited story of a lady leaving behind a purse that was later returned by a government security guard after he noticed it on the camera network while watching her on her commute is seen as an example of public safety, not of spying or overreach. Similarly, a 2018 study found that the proposed social credit system is seen by a majority of respondents as promoting honesty amongst the population. Why? Because its consequences are enforceable by a legitimate entity. Intuitively, it makes sense: the logic is that the government knows what they’re doing because they achieve results and they tell people they know what they’re doing, so it makes sense that the direction they set should be the one to follow for the greatest benefit to the people. This would therefore be the best way to maintain the social order that has been necessary to fuel China’s rise.

But this is where it gets tricky: If a government’s legitimacy rests on maintaining social order, they get to define what that order looks like. In most cases, governments who call for “peace” are really just asking for people to be quiet. People who make noise are seen by the electorate as rocking the boat in an era where stability and unity are critical to national success and are a core part of a country’s identity. If this occurs in a single moment in time, it can be diffused through propaganda and enforced persuasion. But as social attitudes change and evolve (as they do in any society), the government has to be able to continue to reflect the views of those they serve (i.e. they need the support of young people) or they risk losing their perceived legitimacy.

In other words, China needs to remain flexible on what China “is” while simultaneously seeing far enough ahead that it can predict what China “will” be if they want to be seen as knowing everything. Growing the economy has been a simple enough way to straddle both lines. But the more ingrained any social monitoring system gets, the more the technical issues of AI systems will become systemically embedded in China’s government. China’s massive monitoring system could not predict protests in Hong Kong because it is impossible to accurately predict what will drive which changes where in a complex and dynamic network, even with advanced monitoring and predictive systems. AI systems are great for predicting the actions of set-technical systems like machines, but social dynamics are far more dynamic and less quantifiable – it’s hard to know what impact an action will have, if any, and what a response might be to mitigate its potential downsides. Additionally, AI systems tendency to ingrain bias could be a major problem for governments. It could lead to potential suffering or fail to recognize shifts in social norms, reducing legitimacy and creating a feedback loop where the government fails to see or address new trends because the system they’re using to monitor them doesn’t recognize them as such. As one political scientist put it “Machine learning could turn out to be disaster for authoritarianism” if leaders believe the insights it derives are the unfiltered truth. These risks are elevated by the reality that China’s communist leaders are disproportionately technical – 80% of China’s high-ranking political officials are scientists or engineers by background. This means they see the world similarly. But social systems are not about optimization or knowledge advancement, they’re about dynamic interactions between individuals, institutions and culture. Relying too heavily on any one data source will inevitably lead to bias or create blindspots.

China’s push for AI dominance will be supported only by steps that are needed to make the technology more accessible and safer for everyone else: greater transparency in understanding how algorithms make decisions will reduce the risk of blindspots, more common ownership of platforms can ensure knowledge is advanced and decision-making better understood, and standards development that will inform how to best overcome bias within the technical design phase (or offer opportunities to design around it/complement existing systems). The unfortunate truth for authoritarians is that these steps will also play a derailing role in the states ability to control the monitoring systems by democratizing the technology and allowing for greater citizen engagement. In other words: China’s surveillance state needs to become more democratized if it wants to achieve its aim of becoming less democratic, which will unfortunately undermine the whole point of the endeavour. Ironically, it may be this exact push for greater control that ends up landing the country in the news – and proving pundits wrong – for the opposite reason than they expect in the coming years.