The world’s leading publication for data science, AI, and ML professionals.

When Identity Becomes an Algorithm

Meditations on AI, Reinforcement Learning and the Extended Phenotype

Venetian Mask https://pixabay.com/photos/venetian-mask-carnival-feathers-1283163/: Pixabay License
Venetian Mask https://pixabay.com/photos/venetian-mask-carnival-feathers-1283163/: Pixabay License

Discussions on the interplay of humans and Artificial Intelligence tend to pose the issue in the language of opposition. However, according to the thinking of evolutionary biologist Richard Dawkins, tools such as AI can be better thought of as part of our extended phenotype. A phenotype refers to the observable characteristic of an organism, and the idea of the extended phenotype is that this should not be limited to biological processes, but include all of the effects that the genes have upon their environment, both internally and externally.

We are used to defining ourselves strictly by the space we occupy in the physical world. This is misleading. The numbers of non-human cells that occupy our own body outnumber the number of human cells and vast colonies of bacteria swarm within the interior of our digestive tract. Author Robert Svoboda compares the human to a minority government ruling a primarily non-human population. But just as our internal world is largely non-human, our external world is more human than we might imagine. Large parts of our intelligence have already been "off-shored", and exist in the form of computers, cell phones and other devices. These tools act like a new pair of eyes and ears for us– helping us gather information about our environment. Due to our extended phenotype, humans of today are already a very different kind of organism than our ancestors who existed only a few hundred years ago. The degree to which such tools exist outside our body is quiet arbitrary – is a pacemaker more "human" for existing within our body while a laptop less human because it exists outside our body? From the perspective of evolutionary biology, they are both part of our extended phenotype.

If the day comes when we succeed in implanting a cell phone inside our brains or otherwise enhancing our minds with embedded technology, we will likely think at that point of having become partly machine. However this is principally due to the habit of considering a human as only what is contained within them. According to the principle of the extended phenotype, anyone who regularly uses a smart phone is already well on their way to becoming part machine, or rather, the machine has already become part of them. Even now, using your smartphone is likely to have implications for your grey matter, diminishing certain cognitive abilities. Studies have found that those people who consistently use the GPS navigation in their phones have diminished hippocampal activity and less grey matter in that region. The hippocampus is responsible for memory and navigation processes and by using the smartphone for these tasks, one gradually loses the "biological muscles" that previously performed this duty. Evolution is nothing if not thrifty and if we are accomplishing the same function with a different tool than our brain, that grey matter will be repurposed or possibly cease to exist at all. This idea that the brain itself is a survival tool created by the genes is useful analogy for thinking about AI and computers, which are themselves tools created by brains for its own purposes. The full picture of an organism must therefore include all of the adaptations resulting from the genes, including brains, computers, grocery lists and the rest.

While some might fear this transition to an increasingly inorganic phenotype, as inorganic tools begin replacing organic ones, perhaps it will come as a consolation to realize both are part and parcel of our extended phenotype. If a cell phone can remember numbers better than a human brain, then it is likely to be used for that purpose and slowly take the place of the biological tool we were using previously. According to this perspective, we have already been turning ourselves into machines bit by bit for a long time, finding better tools outside of us to do what was previously done inside of us. The location of these tools in respect to our own biology is arbitrary. And as we get better at surgically adding non-biological parts to our organism, such as mechanical joints, organs, and even memory modules, this transition to a non-biological phenotype will become both wider and deeper. In fact, this process of augmenting our biology through surgical instrumentation already has a name, transhumanism, and claims a growing number of adherents worldwide.

Just as we have begun supplanting our internal navigation capacity with the one on our smartphone, and more and more of our memories get stored in note taking apps or by digital assistants, this process is likely to increase with the advent of cutting edge AI techniques such as deep Reinforcement Learning. Specifically, if computers can learn strategic decision making better than our brains, than we are likely to forsake our brain’s abilities for those offered by the algorithm. So long as the algorithm remains our agent, then having it learn a game like chess for you could make sense if it can do so easier and faster than training the neurons in your brain.

In the past, previous types of artificial intelligence were limited in this regard – like the chess playing machine called DeepBlue. They were laborious hand-crafted solutions that served a specific task. Deep reinforcement learning algorithms are far more versatile. DeepMind demonstrated this with the ability of a single reinforcement learning algorithm to master several types of Atari video games. As such, deep reinforcement learning algorithms will likely mark the next wave in the expansion of our extended phenotype.

Games of Unbounded Expertise

One might wonder if games like Chess and Go will cease to be interesting for human now that our brains are all almost equally bad compared with the top AI algorithms. This is unlikely to be the case. One of the interesting aspects of many such games is that there is no upper limit on how skilled one can become. Mastery often depends on how long the algorithm has been training on the task and how good it’s "theory of mind". Demis Hassabis noted that they never found the upper limit of how skilled the Go playing algorithm AlphaZero could become. After 8 days of training, they switched the system off. Given identical algorithms, the person who trained theirs the longest would likely have the advantage in a game of Go, just as the person who trained their brain the longest would be at an advantage in the biological powered equivalent of the game. In other games, such as the variation of Poker known as Texas Hold’em, the skill of a player depends crucially on their ability to read the other player’s strategy, and as the opponent’s skill increases, so does the potential complexity of the game. In this scenario, having a good "theory of mind" about the other player improves ones skill. It may come as a surprise to many, but algorithms with a theory of mind are already out in the wild. For instance, OpenAI developed such an AI system that I have written about in "Have Robots Achieved Consciousness?" In summary, as we export our strategic faculties to machines, we are likely to encounter many of the same competitive scenarios that make biologically powered games interesting to play.

Today, a large segment of the activities that engage human interest exhibit this quality of unbounded expertise. For instance, it seems already overdue to have an Olympics that allows for non-biological enhancements since the current demarcation of where our biological phenotype ends and our non-biological phenotype begins is increasingly blurry. Should someone with a pacemaker be forbidden from participating in the Olympics? What of someone who has had reconstructive knee surgery and replaced their natural tissue with an artificial tendon?

While there is likely an upper limit on certain skills such as folding t-shirts, there seems to be no limit on how good a poker player one can become. Any activities whose complexity scales with the skill or intelligence of one’s opponents will exhibit this quality of unbounded expertise. Little is to be lost by automating activities that possess bounded expertise. Once one has achieved a high degree of skill with such activities as t-shirt folding, we tend to find further investments of time and energy stultifying. Those activities with infinite horizons of expertise on the other hand, are likely to remain important to humans, even as we replace biological portions of our intellect and body with mechanical counterparts.

Those that would bemoan this process, and suggest there is something unnatural about it, would benefit by realizing that we are hardly the only animals with an extended phenotype; and we have already been extending it for a long time. Arguably, since the first human picked up a piece of flint and shaped it into a spear head, or used fire to make food more palatable for digestion, we have been augmenting our extended phenotype. Yet, we sense that with artificial intelligence and particularly deep reinforcement learning, some threshold has been crossed. Up until now, human’s high-octane ape brain has possessed a strategic advantage over every other learning agent on the planet. This is no longer the case. In an increasing number of activities, computers will possess a strategic advantage. Since we have, to a large degree, associated our own motives with the motives of our brains, we are right to be nervous. While the motives of our brains are highly informed by the motives of our genes, they are not identical. A brain can decide it does not wish to procreate, a decision that definitely does not align with the goals of our genes. Therefore, from the moment our genes evolved large brains to look after our interests, there has been the threat of principal-agent dilemmas, or in laymen’s terms, a conflict of interest.

Identity – Who’s Interests Am I Representing?

Now that we are adding one more layer to our phenotype in the form of deep reinforcement learning, if this tool should develop goals contrary to those of our own brains, the situation could get distinctly weird. Questions of identity are always the thorniest – do we represent the interests of our brain, our genes, or of our extended phenotype in the form of some other tools in our employ? Those of a religious persuasion would likely add to this list, the interests of the soul. The resulting dialogue between these different entities is one that can be fairly divisive. And the locus of control between them is increasingly flexible.

Consider that parts of our extended phenotype are already developing interests of their own. Tools that contain advertising are at the forefront of a new type of extended phenotype that introduces the potential for a conflict of interest to exist between a tool and its user. When a tool acts in direct obedience to the brain controlling it, there is no question of a conflict of interest. A hammer, for instance, contains no interests of its own and is in complete subservience to the person wielding it. The Facebook App running on a smartphone is another matter entirely. Your interests in using the Facebook App may depart from the motivations of the app. Facebook’s business model works through advertising, so the motivation of the app is for you to click on one of advertisements it is displaying to you. As author Andrew Lewis has quipped "if you’re not paying for the product, you are the product". While this may seem only a nuisance at the moment, it hides a darker subtext. As the engineers behind such advertising platforms get more adept at manipulating you into clicking things you would not otherwise have done, the chances that you will find yourself sidetracked and turned to shopping or researching products, when you meant to write a thoughtful note to your friend on their birthday are increasing. This is a very important departure from tools of the past.

Previously, tools tended to remain in direct alignment with the interests of their user. With the advent of embedded advertising, software becomes like a Trojan horse, hiding its own agenda. The question of whose interests a tool is serving will increasingly be up for grabs. The story of how this came to be is an interesting one, with important consequences. Travel backwards in time to Silicon Valley in early 1980s, buzzing with young programmers riding the cresting wave of the personal computing boom. Many of these young visionaries belonged to the free software movement. Contrary to public opinion though, programmers need to eat to and these young idealists found themselves at a crossroad – price their software or go broke. Then a solution emerged, enabled by the internet. Software could remain free it would contain advertising. This made way for the profit models of companies like Facebook and Google. Ostensibly, a great deal for everyone, the programmers got rich while distributing their wares without a price tag. But as they say in Texas "there ain’t no free barbecue" and unbeknownst to many users of the software, a catch was lurking in this business model.

Advertising, coupled with interactive software, leads potentially towards addiction and behavior modification. It is almost a mathematical certainty guaranteed by the profit maximization principle. When a piece of software makes money through advertising, it now serves two masters – one being the user, and another being the people who are paying for ad space. The interests of these parties are guaranteed to diverge. If a company makes money by how frequently a user clicks on an ad rather than how effective the product is, then the product’s real purpose becomes getting the user to click on ads rather than accomplishing something useful with it.

Like the interests of the gods and goddesses of old, the interests of a company like Facebook are entirely imaginary, existing only in the collective imaginations of humans. But while such corporations may only exist as fictitious entities, their interests once embedded in silicon chips are very real indeed, and may conflict with the interests of the humans using the software. This is an important point to note as we begin an inventory of our extended phenotype. Consider that it is already difficult to identify the interests of one’s brain versus the interests of one’s genes. While they are closely allied, they’re not identical. Now that our tools are also beginning to have interests of their own, the confusion compounds.

As people identify more strongly with their non-biological extended phenotype, i.e. social media personas and the algorithms that run them, their interests will to a large degree be modulated by the interests and requirements of this extended phenotype. The needs pertaining to maintaining one’s Facebook identity could in fact predominate over the interests of the genes. We already have examples of this in the form of video game players who have forgotten to feed themselves or their family by identifying so strongly with the goals of characters within the game.

One of the most crucial questions to consider going forward is where we will come to place the locus of control for our identity. Will the tool chest of the genes stage a comeback in the form of genetic technologies like CRISPR, reuniting us with our biological containers, or will we continue the long march towards a non-biological extended phenotype, outsourcing more and more decision to computers while gradually replacing our biochemical algorithms with inorganic ones. In this light, deep reinforcement learning would seem to be a tremendous leap forward for the inorganic extended phenotype since it will enable this inorganic extended phenotype to tackle problems that previously only our brains could.

Whats more, if human life is indeed a drama of decision making, as our art, religion, and even legal system treat it to be, then it follows that machines possessed of reinforcement learning abilities are in effect, moral agents. Either something will have to shift in what we consider a moral agent to be, or we will have to expand our thinking and treatment of such machines to encompass moral agency. Untangling the legal responsibilities and protections that follow from this chain of logic is likely to prove a daunting yet unavoidable task. Right now these are mere fringe issues with little relevance outside academia, but they are almost certainly destined to become questions of enduring importance. What’s more, the window for making meaningful progress on them is likely to be far shorter than previously believed.

Consider the increasingly common phenomena of Facebook pages or email accounts that survive their owner’s death. If such digital personas were endowed with reinforcement learning algorithms allowing them to continue responding and adapting to the stimuli they received through posts, messages etc., perhaps in accordance with the style and goals laid down by their original user, then in a very real way, one’s inorganic extended phenotype could persist long after ones biological death. Should such inorganic phenotypes be afforded any legal protection? If we were to remove a person’s brain and keep it alive inside a computer such that it could continue to write messages and communicate, we might consider that person still alive in some sense and offer them some legal protection. The comparison is not altogether unreasonable and such bizarre questions of identity are likely to be thrust upon us much sooner then we realize. While our present phenotype could be described as a kludge of biochemical algorithms assisting in the survival and replication of the species, our future phenotype is likely to resemble a kludge of inorganic algorithms, whose purpose and design will be far more variable than those dictated by the strict terms of evolutionary fitness.

While we currently give our extended phenotype a limited degree of autonomy, i.e., we may allow Google to generate automated responses to emails or schedule appointments for us, that autonomy is growing. There is certainly a danger in off-shoring too much of our decision making to faculties not wholly owned by ourselves. If Google or Facebook owns large portions of our extended phenotype, then we must add to the locus of our decision making the interests and wants of the corporations owning our extended phenotype.

A careful inventory of one’s extended phenotype, and various interest groups whose influence or control it is under, is perhaps the most under explored region of our education today. When we walk into a friend’s house and ask for the wifi password, whose interests are we serving – those of our genes, our brain, or of our extended phenotype in the form of the Facebook app? By failing to realize that are all different entities cohabiting within our extended phenotype, we easily fall under the thrall of the one with the loudest agenda. Certainly, the brain and body must sign off on the order to ask for a WIFI password, since they represent choke points in the decision process. However, the real string puller may be the Facebook app when we find ourselves diverted to clicking on ads after we get online.

Like a type of fungi found in the Brazilian rain forest that spreads by invading an ant’s body and turning it into a zombie, marching the fungi to a new location before dying, so we may find ourselves co-opted by manipulative software and reprogrammed to do its bidding. This is certainly not the future we would hope for, but it may be the unintended consequence of certain profit models. Going forward, it will be increasingly crucial to make a sincere accounting of our extended phenotype such that we avoid coming under the thrall of algorithms designed to steer us towards goals we would not otherwise have chosen.


Related Articles