Programmable Loyalty
How automation can create the next dark ages.
So nowadays there is a lot of doomsday-scenario talks about artificial intelligence and the rise of a machine race that will use humans as raw materials for their latte. Like most people, I also have my two cents to add to this discussion, but the way I look at it, it’s not just the machines that are a threat. The companies that are willing to make them also are. In the event that we give rise to a powerful superintelligence, even if it is controllable and friendly, whoever has control over that becomes a major threat.
A staple of cyberpunk is that companies control as much resources as national governments. In fact, many companies supplant their national governments altogether and simply install a tyranny whereby the population is given just enough to serve the objectives of the company (which, at this point, has become a sort of autocratic or oligarchic government in and of itself, entirely disconnected with the national identity of its people).
Though this seems ridiculous by our modern standards, I believe there is at least a veneer of possibility. You see, there are several reasons to believe that a company could never wrestle control over population centers and infrastructure from a federal government because the government has armed forces and the backing in legitimacy of its citizens. As long as the government endures, it is more likely that citizens will be loyal to their nations than to the company where they work.
This monopoly of force and loyalty preferences can both be circumvented by the employment of artificial intelligence and automated decision-making. In order to see where this is going, let’s first analyze the darkest, most drastic example of what I’m talking about and then go several notches down to a level closer to what we can expect to happen in the next decade or so.
The Auto-Tyrant
Automated production lines are effective. Much more effective than human-controlled ones. We have seen it stated many times over. In fact, automation’s entire job description is about creating abundance at low cost via using raw materials and energy supplies as efficiently as possible.
Nowadays a factory still requires a lot of human hands to make things happen. Soon or later, however, we will have a factory that operates entirely on its own, requiring the supervision of one, or maybe a handful, of high-level tech and engineering overseers. Machines are growing more autonomous as we speak and it’s only a matter of time before they can navigate well the decision-space required to accomplish objectives within a narrow activity such as bending metal, riveting things to other things and cutting steel pipes.
The Auto-Tyrant is the ultimate level of this. It’s, say, a thousand of these factories controlled by one single person. Everything else is AI, automated and fully under the Tyrant’s control. In this setting, the Tyrant can conjure anything to fabricate, limited only by the permits and licenses it can obtain from local government. There are no employees to tell on him if the activities become a little unsavory (like manufacturing weapons, or robotic soldiers).
A company that is run by an auto-tyrant isn’t just able to become traitor-proof, it can also be made extraordinarily efficient. A factory such as this never stops, never tires, never goes on strikes, never requires food or water. It’s only needs are energy, maintenance and raw materials/components. Production-wise, it is impossible to outclass a factory such as that one, so there will be huge economic incentives to get these things running as soon as possible.
Entire supply chains could potentially be set up that automatically transform raw materials into just about anything without any substantial human intervention along any phase of the process. Once you learn to automate a factory you might have the keys to automate maybe any factory (or at least a wide spectrum of factory types usually employed by heavy industry).
These auto-factories are traitor-free. Loyalty of the system to its master need not be earned through good company policies, extra payments, production incentives and goals… None of that. Loyalty can be programmed and made absolute. An auto-tyrant can control as much production output as a country, all by herself, without any danger of revolt against an ironclad mandate directly programmed in the skulls of her “employees”. The moment that creature decides to fabricate an army, it might be able to take control of a small country where some of its factories are.
At that point, the company might not even become the government itself, but it might simply establish whatever government it wants under the threat of its robotic armies should the local population fail to comply. It is arguable what other steps it would take after this but there you have it, a country made subservient to a technocratic feudal lord because of automation.
Something more real
Okay so maybe an technocratic tyranny built on top of robotic soldiers is a bit far-fetched, but the point of the previous section was to make a thought experiment. That is the kind of power a single individual can potentially control if automation becomes as pervasive and as efficient as it is poised to become in the coming decades. We might, in the far, far future, watch as great houses of technocractic inventors vie against one another using automated armies for the control of the planet (or the stellar system, or the galaxy….). If it comes to that point, then the scions of these great houses will live in luxury and opulence impossible to describe. But it’ll be pretty much game over for everyone else.
Having complete control over production means, state-of-the-art armament and the absolute, unwavering loyalty of an automated hierarchy capable of projecting power across vast spaces, these people would have no need for regular individuals except for their own personal agendas like choosing romantic partners or the creation of genuine human art, or just to keep them company.
The quality of life of the regular person would be the mercy of the whims and desires of the Tyrant under whose banner they lived. Don’t expect much. I mean, even feudal peasants had direct, measurable economic value to their lords, and look at their lot in life. Imagine how much worse it would be for people whose effort and expertise is entirely not needed. They would be worse than dirt.
Any innovation, art and other high-level human things would likely be restricted to the members of the great houses. It is possible that they would find companionship amongst themselves and leave the peasant rabble to rot and wither away from the planets they controlled. Or they might actively pursue and eliminate the lower levels of humanity in order to “swipe the planet clean from this polluting infection.”
How far are we from this dystopic nightmare? Maybe very far, maybe not so far. It’s hard to tell because the very pace of technological advancement grows everyday and the technology of tomorrow is already on the making, away from our prying eyes. We can only look at what we have now and extrapolate.
But even if you don’t have entirely automated armies and factories. You don’t need them. All you need is that the Tyrant has the keys to the realm. If the tyrant controls how and when machines are turned on and operated, then you don’t need the machines to be entirely autonomous. Even simpler machines that require some level of human oversight will add to the balance of power in favor of the factory owner.
Every increment in technology that reduces the number of people required to control and operate phases in the long supply chains required to turn raw materials in high-value manufactured devices is good news for an autotyrant-wannabe.
The operators of an autofactory are more valuable to their master than the regular operators. Because of that, and because less of them are needed, it is reasonable to assume that the factory owner can pay them a pretty hefty salary. In a struggling economy, this buys loyalty. In the future, when the company goes ahead with its coup, these people will be overseeing the fabrication of robot-soldiers without flinching because they know that they are “in”. They might not be the royalty, but they are the king’s retainers and they have landed on a job on the good side of the tyranny. None of them would rationally risk this job, particularly if they have families.
It’s the same with the armies. A fully autonomous army is not needed. What is needed is the capacity to efficiently fabricate high-quality automated equipment that can be operated by a skilled individual. The company need only find skilled people that are unemployed, sick of the government and living through economic turmoil. No hard to find. Even if you can’t direct entire divisions of armored vehicles with the push of a button, maybe you can automate enough to allow a single corporal to guide a small squad of robotic drones.
Any advancement in automation that renders human effort obsolete is progress towards this technocratic nightmare because it means that the loyalty of less and less people is required for an autotyrant to take control of vast amounts of resources and assets. The Tyrant would only ever have to reward and buy the loyalty of a small group of technical experts and machine operators and since the size of the bounty to be seized is so large (and entire country worth of raw materials and production efficiencies verging on the infinite abundance), then the Tyrant may promise lives worthy of kings to each of her supporters, making it more likely that people will ditch their national allegiances and side with her in bringing a new world order.
If the Autotyrant can get a majority support, the majority might allow the dissenting minority to be purged or they might escape, no matter, the Tyrant has a hell of an offer for anyone with the right skills that want to work for her. It won’t be for everyone, but it might tempt enough people to allow a Tyrant to replenish the stations of those who left or were purged.
I have no idea where that train of thought will lead if you go at it hard enough, but it can get pretty crazy pretty fast. I would advise against taking this at face value, for me it’s kind of a parable about the dangers of automation. Something to muse about but not really possible. I mean, that can’t happen in real life, right? There are powerful people fighting one another for control, each hampering the other. It seems unlikely that a few of them would manage to reach this level of control and maintain it. The moment a company did a coup, countries would fall over it like locusts, I think.
But then I see something like this and this, both of which tells us that armed forces around the world are seriously considering developing and manufacturing robots for military applications using AI to control them properly. Who is going to manufacture those robots? Can we be certain they don’t have a backdoor? Are we trustworthy of their allegiance? This is closer to auto-tyranny than we are now. And every step closer makes it more likely that a small group of people can pull something like that off.
The rise of superintelligence would surely put all of these concerns into overdrive. A machine such as that would stack the odds in favor of its master until the stack reached the edges of the Hubble volume. But even if our AI remains narrow but highly efficient, multiple systems can be combined to generate a broad-purpose mid-range automaton that is neither too narrow in its capacities nor too general in its intelligence. That type of mid-grade AI may be enough to take automation levels past the autotyranny threshold. So it is important that we understand who controls that technology and who is watching them.