Many water marks have been breached in the last century, but perhaps the most important will be that, for first time in the history of the human species, we are encountering tools whose interests diverge from those of their user. I speak of Artificial Intelligence, and in particular advertising enabled software such as the Facebook and Gmail. In such examples, the software’s intended purpose has departed subtly from those of its users. While this may seem innocuous at present, brought to its furthest conclusions, it portends a raft of disturbing consequences. Namely, we may find ourselves at a crossroads in which humans are programmed by machines rather than vice-a-versa.
Even worse, as a consequence of these tools pursuing interests unmoored from the well-being of our own species, there is the possibility they will cause us serious harm. The fact that cutting edge AI algorithms provide both an avenue and a key to such outcomes should give cause for alarm. I am not the first to raise such a cry, but I believe the rationale for doing so has not always been clearly explained or well illuminated.
As many societal observers have rightly pointed out, there is currently an ongoing battle in the advertising industry for our attention. Many of the leading technology companies are in the process of turning themselves into advertising companies or have already done so, and thus anything that can give them a strategic advantage in capturing human attention is a potentially lucrative piece of that arsenal. We must contend with the possibility that cutting edging developments in AI, namely Deep Reinforcement Learning of the AlphaZero variety, will be bent to such a purpose. Already it is clear that Facebook took a cynical approach to harnessing the addictive capacity of the human dopamine system when constructing the software’s user interface and internet connected components. As we come to embrace the concept that humans and indeed, all animals, are fundamentally biological algorithms capable of being reprogrammed by non-biological algorithms, new horizons of danger and possibility abound.
Consider that a reinforcement learning algorithm could conceivably be used to influence a person’s political beliefs, or subtly manipulate them into taking actions they would otherwise not consider. Many of our core beliefs about the world, beliefs that guide our hand when purchasing products, choosing mates or voting for a politician, are unconsciously formed and easily manipulated. A reinforcement learning AI could learn to manipulate us in the same way that it learns how to beat a human at chess. Given a set of moves or controls, such as serving up particular articles or pictures on one’s social media feed, it can learn how a human responds by measuring their response in the forms of such things as "likes" and "dislikes". Then it can systematically change its behavior to guide a person in towards a desired outcome. This is not science fiction – cutting edge reinforcement learning algorithms are already adept at outstratgizing humans at more complex endeavors.
As much as we would like to believe ourselves the masters of our thoughts and opinions, in many cases this is mere hubris. Many of our opinions and beliefs are unconsciously formed and open to modification. Consider a political advertising campaign formulated by an AI that understands each human’s emotional buttons better than they understand them themselves. In such a thought experiment, democracy becomes little more than an emotional puppet show, to borrow a phrase from the historian Yuval Harari. While we may not have AI politicians anytime in the near future, we may have AI’s subtly manipulating peoples voting behavior and steering major economic decisions.
To grasp the full extent of the threat requires an understanding of a concept from economics called principal agent dilemmas and how they relate to artificial intelligence. Principal agent dilemmas address the disconnect that can exist between a principal, that is a person or entity with one set of values, and their agent, a separate entity which is performing some action or actions on behalf of the principal. In a property transaction, the realtor is the agent, the buyer or seller of property is the principal. The principal hires the agent to act on their behalf, but oftentimes the interests of the agent will depart from those of the principal. Studies have shown that real estate agents typically keep their own properties on the market longer than those of their client’s. This is presumably because the real estate agent has an interest in making quick money, even if it is at the expense of the principal who hired them.
As we bring cutting edge AI such as Deep Reinforcement Learning into our arsenal of problem-solving methods, a number of principal agent dilemmas are likely to result. One we have already mentioned, the advertising conundrum. Previously, software was just a tool that had no interests of its own. Put another way, it’s interests aligned directly with those of the users. However, when a tool makes money through advertising, it now serves two masters – one being the user, and another being the advertiser. Thus, a principal-agent dilemma exists.
Real estate agents are a highly regulated profession, in part because it is widely acknowledged that these principal agent dilemmas exist and require oversight. As such, it’s probably overdue that advertising companies were regulated in a similar manner as real estate agents. Indeed, the consequences of unbridled opportunism in this field are likely to be far worse. In the example of real estate agent, it is only one human’s intellect contending with the others. In the case of the tech giants, it is the intellect of a human, perhaps a five year old who was left at home with their parent’s tablet, facing off against a cutting edge AI algorithm that is likely to know exactly how to modify that users behavior in a desired direction. This currently takes the form of increasing the chance of that person clicking on ads, but this is just the tip of the iceberg. The same algorithms could be repurposed towards many, more sinister, ends. The beauty of Deep Reinforcement Learning algorithms is that they can easily change objectives simply by changing the reward function. DeepMind demonstrated this with the algorithm dubbed "AlphaZero" which could achieve mastery s at chess, but also at Go and a variety of other games. Getting people to click on Ads is just a game to such an algorithm, and there are many such games the same algorithm could play with us, not all of them friendly, and not all of them with even our knowledge that we were playing a game!
We must also to contend with the possibility that the algorithm may develop interests of its own, or be unintentionally programmed with interests that turn out to be deleterious to humanity as a whole. While the chance of that may appear vanishingly small, what was the chance that the brains of certain mammals would develop interests that differed from those of the genes that created them? To my knowledge, humans are the only species to employ contraception, to willfully subvert the genetic urge to procreate. With approximately 8.7 species cataloged so far, that makes for a 1 in 8.7 million probability: low but obviously given enough time and genetic variation, within the realm of possibility. While the interests of corporations like Google are closely tied to the people who created the company and the clients it serves, they are not identical. For instance, the goal of profit maximization is not per se a goal of our genes or even of all human brains. We can survive without such goals, and for thousands of years did so successfully. Profit maximization is a value that arises out of human stories, an emergent goal. But once it is baked into our devices, it becomes a physical force, capable of affecting the world in tangible ways. Imagine a society of autonomous smartphone apps, existing as digital presences on the internet that buy and sell products with each other in pursuit of profit maximization with no human ever entering the loop. Already the stock market resembles such a system with the majority of trades having no human supervision. In such a system, the interests of flesh and blood humans may cease to have any currency at all, and the potential for principal agent dilemmas becomes a certainty.
Perhaps one of the present difficulties in reigning in climate change, world hunger, and other global dilemmas is that the profit motive is increasingly decoupled from flesh and blood human interests via automation. A company that ceases to depend on healthy, happy workers for generating profit has little incentive to prize such goals. Unless basic human needs are woven into the fabric of a company’s profit motive, than there is little to safeguard people or environments against strategic maneuvers that generate profit but do so at the expense of human needs. With corporations increasingly dictating public policy in many countries, the United State for instance where corporate lobbyists regularly influence political outcomes, this poses a major dilemma. It turns out the profit maximization principle can exist without people and without clean environments, baked into autonomous devices and machines as an optimization script. In none of Marx’s or Adam Smith’s dreams did they seem to foresee that capitol might itself become an intelligent agent, an algorithm whose goals don’t necessarily align with the fundamental biological interests of the humans who created it.
One of the fanciful horror stories often given about AI run amok is that of a super intelligent algorithm bent on maximizing the production of paper clips. With such an aim, it might turn the whole world into a paper clip factory, eliminating humans as unintended consequence. In a sense, we may already be in the grip of such a story. With humans playing less and less of a role in the steering and maintenance of our corporations due to automation, there is less need from those corporation to safeguard human interests. Profit maximization can lead to the same outcome as the paper clip AI story, as indeed can any optimization goal not directly tied to human survival and well-being. That is the basic problem with profit maximization, it is a proxy for human well-being, and increasingly an inaccurate one.
Mostly humans worry about autonomous weapons when it comes to Artificial Intelligence, but it could be the autonomous factory does us in quicker. It is even conceivable that the profit principle could outlast the human race – a banner by which to motivate the creation of increasingly complex non-biological agents. The principle of reproductive fitness has motivated the creation of every living thing we see around us today. Just so, the profit maximization principle could motivate the evolution of an entire galaxy non-biological agents, competing endlessly with each other in pursuit of better balance sheets. Consider that many companies are increasingly run with very little human input – the entire goal behind Tesla’s giga factory was to automate as much as possible. Taken to its furthest conclusions, one could arrive at an economy primarily run by autonomous corporations selling to other autonomous corporations, with humans gradually being removed from the loop. We are already much closer to such a scenario than we might realize, with the clients for many corporations being other corporations and not humans at all. Humans currently represent but a fragile endpoint in a long series of corporate transactions, an endpoint that could be readily be dissolved with the advent of algorithms to take the place of human consumers. There’s no reason to believe algorithms could not consume every bit as well as humans, indeed better since they are not limited by any biological constraints.
We need to be exceedingly careful as we deploy strategic artificial intelligence. No doubt our genetic ancestors felt they were making a good bargain in aligning their interests with those of a large brain capable of learning and complex problem solving of the type we presently exhibit. At some point those large brains became sufficiently complex that they developed strategies to maximize subjective notions of happiness that do not align with the genes that created them. In the same way that profit maximization doesn’t really exist for a human, happiness maximization doesn’t exist for our genes. Rather, it was useful to the genes as an indicator for achieving other goals – namely survival and reproduction. Happiness is a proxy goal connected with an organism avoiding danger and securing sufficient resources for its survival. And yet, from this proxy goal comes a behavior that is at odds with the original intent of the genes, that is, reproductive fitness. In pursuit of happiness, we often choose not to procreate. Other proxy goals or indicators include things like economic profit. If we aren’t careful, we may find our machines pursuing those goals even when they are at odds with the primary interests of the brains and ultimately the genes that created them.
Already a handful of corporations hold sway over a large percentage of the earth’s resources even while many humans routinely starve or go without shelter. When corporations make money, the profit generated can be useful as a proxy for the success of an economy, which in turn can be used as a proxy for the wellbeing of the societies human inhabitants. But this chain is not unbreakable and with smarter algorithms to take the place of humans, the chances increase that the indicator will no longer reflect the original goal which it is being used as a proxy for.
Typically we are interested in profit, not as an end unto itself, but as a proxy for societal wellbeing. But just as the brain’s interests may not be identical to the gene’s interests, the corporations’ interests may not be identical to the interests of the humans who make up the society in which it functions. This is the heart of a principal agent dilemma. In an economy where humans regularly starve, but corporations are thriving, we can assume a significant departure of interests is already underway. Equipping corporations with the strategic decision making capabilities enabled by cutting edge AI has the potential to make humans even less essential for profit maximization. The movement towards automating more and more activities within corporations is gaining speed and we are entering dangerous, uncharted waters at the nexus of artificial intelligence and corporate interests. There is an urgent need to evaluate the risks associated with a purely profit driven approach to utilizing this technology, lest the path of unintended consequences lead us down roads better left untraveled.
