Irrational AI

Machines are rational, and humans are irrational, right? Wrong. Both humans and machines are irrational and for strikingly similar reasons. In this blog, I’ll tell you why, and how it’s not necessarily a bad thing.

First of all, what do I mean by rationality?

Logical decisions, based on data are rational decisions. Ideally, to make the most rational decision you want a perfect data-set. A data-set which is full, accurate and unbiased. You want to process this data logically, updating the probability of each outcome with each new piece of data.

So, why are humans biased?

Our human brains evolved for thousands of years to eat, sleep, reproduce and defend itself from threats, with the occasional requirement to use some higher reasoning to steal food from others and defend its own from being taken. Society, language, and complex inventions are a very recent thing in our evolution, and it’d be unfair to expect our brains to adapt so quickly to the abstract or highly mathematical thought to make rational decisions.

Making rational decisions requires an accurate and full dataset and a purely mathematical decision making. Our minds are not capable of this. To be useful with the processing power available to them, and within the time constraints that our actions require, we have evolved to make a lot of approximations — This is where we find bias.

Even if we have a perfect dataset humans struggle to make rational decisions. Imagine for a moment you have had a broad spectrum blood test, and your doctor tells you that you have tested positive for a very rare disease which impacts 1 in 1,000 people. The test itself is 97% accurate. So, what’s the chance you have the disease?

A lot of people will jump instantly to the 97% accuracy of the test and think it’s 97% likely. In truth, the probability is only 3%. The reason for this is that it only actually impacts 1/1,000 people, and if you tested 1,000 people you would expect 3% to receive false positives as the test is 3% inaccurate. So we would have 30 test positive, but only 1 actually with the disease, so the potential of you having the disease after one test is roughly 3%.

Eliezer Yudkowski makes a great example of how time constraints play into this — Talking about a tiger. When we see a tiger, we don’t think ‘hmm, that creature is yellow and stripy. Other yellow and stripy creatures in the past have been described as tigers, and I have been told that tigers have a high probability to AAAAAH chomp chomp chomp’. Instead, we see a flash of yellow, the rough shape rough shape, our brain completes the pattern and we run.

Bias, as Eliezer says, is not something we layer on top of a purely rational mind, it is our whole decision-making process.

So, why is AI irrational?

Theoretically, AI has the potential for rational decision making. Where human brains have hardware constraints that we may never be able to overcome it’s comparatively easy to add more processing power to a computer. Once we have discovered the right algorithms to make a decision (I’m not for a moment suggesting humans are good at making fair and unbiased algorithms, but the scientific method should give us some hope), we can be sure that an automated system will always follow those instructions and rules, every time, without fail.

Pragmatically though, AI is quite unlikely to be rational. That complete, accurate and unbiased dataset that rationality requires? It doesn’t exist. That vast processing power to ingest and make decisions based on that dataset, or the best dataset we can come up with? It’s expensive. The only way that we would achieve anything close to a rational AI or automated system is through brute force, and brute force is going to be slow and very expensive.

Businesses are motivated first and foremost by profit. If AI or automation is being considered to replace a historically human run process the business case made to adopt the technology will fundamentally be about how it improves profit. Even well-intentioned people, determined to make rational and unbiased AI will have to convince their peers, putting together a business case which may well look something like — ‘Consumer spending on ethical products is increasing, by building ensuring that our automation promotes equality (seriously, read automating inequality by Virginia Eubanks) we can increase sales and profit’. I would not be surprised to find out that there are businesses in the world who willingly break the law because their unlawful conduct makes them more profit than it costs them in fines. Either way, an argument towards profitability is only going to increase the chance of success for a rational or unbiased product.

If we begin to look beyond more standard automation, towards Artificial General Intelligence, we know that the project ahead of us will be challenging and expensive. One frequently suggested path towards AGI relies on the ‘law of accelerating returns.’ You start by building a narrow intelligence. Something which might be better or at the very least more cost-effective than a human at one specific task. Use the power that brings — Whether it’s the type of reasoning that narrow AI is capable of or the resources you can make from it to build a second, more advanced AI. Rinse and repeat until you have yourself an AGI. The trouble here is that we will be encouraged to make irrational AIs at these stages. As we are limited in resources, data and intelligence at the early stages developers will need to cut corners, finding sometimes elegant, sometimes less-so ways of generalizing problems to get to their next iteration. If we see that law of accelerating returns resulting in a fast and hard takeoff of AI, then it may be too late when we realize it’s no longer in our control, and not a truly rational being.

A simple solution to this would be for the business to slow down, but this issue is similar to a prisoners dilemma, particularly if we are beginning to look at artificial general intelligence. The most beneficial outcome for society is to develop safe, and value aligned AI. I may have control over the system that I am developing, but I have little control over what my peers or competitors are doing. If I am unable to trust their actions, I may loosen my standards for safety, deciding that even if I am cutting corners just like my competitor I believe I am more trustworthy and will likely cut the right, or least damaging corners. Otherwise, I may take a slightly more destructive approach, assuming that an uncontrollable AGI will spell the end for humanity and if someone is closer than me, they’re more powerful than me, and if I can’t beat them, I might as well join them.

Do we actually want a genuinely rational AI?

Mainly if we are looking towards Artificial General Intelligence (AGI) and Artificial Super Intelligence (ASI), we hear conversations around value alignment, control, and ethics. All of these are steps away from rationality and towards a kind of bias, and that’s a good thing.

First, value alignment. What we mean here is explicitly aligning the AIs goals and values to those of humanity. We might not know today precisely what those values are or whether they will be the same any length of time from now, but it does actively push a bias towards what is valued by humans, not what’s valued by other creatures on earth or more directly beneficial to the AI itself. This doesn’t mean that the AI will disregard all other life in the universe, as humans may well value other life (however poorly we show it!).

Ethics is interesting too. I do not think there is any universal ‘a priori’ ethics that the AI could somehow discover. Rather, I think ethics is something humans create and give meaning to, and it’s the average standard of right and wrong within a group of people. For this reason what people have considered ethical has developed throughout human history as our needs and priorities have changed.

So, why automate?

Just because the system isn’t perfect, doesn’t mean it’s not useful. Let’s not forget that humans too are biased. If we are to build a system 10x faster than a human, and also 95% less prone to biased outputs then we have both decreased the volume and proportion of bias in the world — We just have to make an assumption here that we can do this profitably; otherwise we’re unlikely to sell many.

It’s also vital to detach ourselves from a desire to have a human to blame. If we were able to release a driverless car today which is as safe as a human driver there would still be a public outcry if 3,000 people died on the first day of the switch-over. Even if that number is 2,000, even 100 deaths in the first day the reaction would no doubt be adverse. In truth, humans are causing slightly over 3,000 deaths per day on the roads. If our motivation is indeed to reduce faulty output and limit deaths, we should automate.

Let’s not forget the utility that less than rational reasoning can bring. Creating useful outputs, rather than the perfect output but far too late. We must automate mindfully, finding instances where it is viable for us to both reduce biased outputs and increase profitability and take caution where we are being pushed wholly towards speed and away from fairness against our own best judgment.