PODCAST

Existential risk from AI: A skeptical perspective

Melanie Mitchell on the reasons why superhuman AI might not be around the corner

APPLE | GOOGLE | SPOTIFY | OTHER

Editor’s note: This episode is part of our podcast series on emerging problems in data science and machine learning, hosted by Jeremie Harris. Apart from hosting the podcast, Jeremie helps run a data science mentorship startup called SharpestMinds.

As AI systems have become more powerful, an increasing number of people have been raising the alarm about its potential long-term risks. As we’ve covered on the podcast before, many now argue that those risks could even extend to the annihilation of our species by superhuman AI systems that are slightly misaligned with human values.

There’s no shortage of authors, researchers and technologists who take this risk seriously — and they include prominent figures like Eliezer Yudkowsky, Elon Musk, Bill Gates, Stuart Russell and Nick Bostrom. And while I think the arguments for existential risk from AI are sound, and aren’t widely enough understood, I also think that it’s important to explore more skeptical perspectives.

Melanie Mitchell is a prominent and important voice on the skeptical side of this argument, and she was kind enough to join me for this episode of the podcast. Melanie is the Davis Professor of complexity at the Santa Fe Institute, a Professor of computer science at Portland State University, and the author of Artificial Intelligence: a Guide for Thinking Humans — a book in which she explores arguments for AI existential risk through a critical lens. She’s an active player in the existential risk conversation, and recently participated in a high-profile debate with Stuart Russell, arguing against his AI risk position.

Here were some of my favourite take-homes from the conversation:

• Melanie is skeptical that we should worry about existential risk from AI for several reasons. First, she doubts that we understand what intelligence is well intelligence enough to create a superintelligent AI — and unlike many AI risk advocates, she believes that without a sound understanding of intelligence, we won’t be able to make genuinely intelligent systems.
• A second reason for her skepticism: Melanie believes that intelligence can’t be separated from socialization. Humans arguably evolved much of their intelligence through social pressures, and the development of human intelligence from birth revolves around social interaction. Because AIs will ultimately be built with the aim of delivering value to humans, Melanie believes that they too will be “socialized”. As a result, she argues that genuinely intelligent AI systems are likely to pick up “common sense” and “ethics” as a byproduct of their development, and would therefore likely be safe.
• However, Melanie does acknowledge that there’s risk in developing AI systems that are capable of coming up with dangerously creative solutions — ways of achieving the goals that their human programmers might not anticipate or desire. While she agreed that this could be a source of serious risk, she disagrees with the characterizations of AI risk offered by people like Stuart Russell, who, she argues, wrongly assume that intelligence can be developed without socialization.
• While Melanie doesn’t see existential AI risk as something worth worrying about, she does agree that significant non-existential risks might arise from AI technology in the near-term. Malicious use, accidents, or the deployment of AI-powered automated weapons systems could all present us with serious challenges — and Melanie believes there’s reason to call for more AI regulation on that basis.
• Melanie is concerned about the speed at which AI technology is developing, and doesn’t think laws and regulation will be able to keep up. For that reason, she believes there’s a greater burden on researchers to ensure that they’re doing what they can to guide the evolution of AI technology in a safe and positive direction.

You can follow Melanie on Twitter here, or follow me on Twitter here

Links referenced during the podcast:

• Melanie’s book is here.

Chapters:

• 0:00 Intro
• 1:07 Blind spots in AI capabilities
• 9:00 Humans and catastrophes
• 11:44 Understanding intelligence
• 17:58 Humans and emotion
• 23:09 Existential risk
• 26:32 Necessity vs. Sufficiency
• 28:42 Turing test
• 34:19 Vulnerability to malice
• 40:02 Adapting society to changes
• 44:04 Wrap-up

Please find the transcript below:

Jeremie Harris (00:00):
Hey everyone, Jeremie here. Welcome back to the Towards Data Science podcast. And today’s episode is going to be a little bit special because we’re talking to Melanie Mitchell, who apart from being the Davis Professor of Complexity at the Santa Fe Institute and a professor of computer science at Portland State University, is also a prominent author in the AI safety space and a prominent skeptic of a lot of the AI existential risk arguments that we’ve explored on the podcast here. As many of you will know, especially if you’re long time listeners of the podcast, I myself am actually quite concerned about existential risk from advanced AI systems, which is precisely why I found this conversation so useful and fruitful. It’s always great to talk to people who disagree with you, especially if they’re as thoughtful as Melanie is.

Jeremie Harris (00:43):
Finally, if you’re curious to dig deeper into Melanie’s views on AI existential risk after listening to this podcast, I do recommend her 2019 book, Artificial Intelligence: A Guide To Thinking Humans and the podcast that she did at the Munk Debates with Stuart Russell. So without other way, I’ll step aside and let you enjoy the conversation. All right, well, Melanie, thanks so much for joining me for the podcast.

Melanie Mitchell (01:05):
Oh, I’m thrilled to be here. Thanks for inviting me.

Jeremie Harris (01:07):
I’m really excited to have you here. Your 2019 book, Artificial Intelligence: A Guide For Thinking Humans offers a perspective I think that’s quite different from many of those we’ve had on the podcast so far. I think you’re really one of the most interesting proponents of the sort of skeptical side of the AI existential risk argument, which is why I’m so excited to have you here today. Now, AI safety is a big topic. I think there are a lot of places we could start, but I’d like to begin with your assessment of where we’re at today in AI, what are the things that AI can do today that maybe you find impressive, but also what are some of the blind spots that we have as AI capabilities are concerned?

Melanie Mitchell (01:47):
So AI has obviously excelled in a lot of re relatively narrow domains, speech recognition, machine translation, and by narrow, I mean, sort of narrow is compared to the generality of human intelligence, right? So each of these systems is able to do a particular task and it can do it very well, but it’s the challenging thing is getting machines that are able to take their knowledge on one kind of task and adapt it to another kind of task, or to be able to deal with new situations that are sort of outside of the distribution of their training data and so on. So these are well known challenges for the field and they do produce some, what people call brittleness, meaning that the machines that are trained to do certain tasks can have unpredictable failures, very unhuman like failures, and this can lead to risks. It can lead to risks like we see in self-driving cars, for example, that can make errors that can say see a picture of a person on the back of a bus and think that it’s an actual pedestrian, or that recognize, think that a sideways semi-truck is actually just the horizon or whatever the different errors are, errors that humans wouldn’t make. Of course, humans make different kinds of errors when driving.

Melanie Mitchell (03:21):
But this just gives you an example of the kinds of risks that make it hard for us to trust these AI systems with our lives. And then there’s also the vulnerability to adversarial attacks that we see that makes using these systems in any kind of life critical situation kind of difficult right now, because these systems do have vulnerability.

Jeremie Harris (03:55):
Would you be more impressed with AI technology if its failure modes were more human like?

Melanie Mitchell (04:00):
I would be more impressed if its failure modes are more human like, because then I would have a better idea of when to trust it and when not to trust it. That would make it easier to sort of figure out how to use it safely.

Jeremie Harris (04:16):
Exactly.

Melanie Mitchell (04:16):
Humans have their own failure modes and biases and so on, but we’ve kind of developed infrastructure if you will, to deal with some of that. And now we’re taking AI systems and we’re trying to re jigger our infrastructures to deal with their kinds of failure modes. So it would be better if the machines were able to better be like humans, obviously to…

Jeremie Harris (04:43):
It’s interesting what that all implies about the way that these AI systems are reasoning about the world, if that’s the appropriate word. The fact that those failure modes are so different, sort of seems to hint that there’s something else going on. Maybe that’ll be actually a good foundation for the next question I’m going to ask here, which is about the AI existential risk argument. So I mentioned earlier, I mean, you really are a well known AI existential risks skeptic, though you certainly have highlighted a number of risks with the technology. What’s your take on the arguments for existential risk? In your model of the world, why are people concerned about existential risk and then what is it that those people are missing?

Melanie Mitchell (05:22):
So my understanding of some of the people who are worried about existential risk is that… Here’s sort of my version of their argument. We don’t know how soon we’re going to get human level AI or super intelligent AI and there’s some worries that such a system could make decisions that affect the existence of the human race. There’s the existential risk part because they don’t share our values, or have our kind of common sense abilities. But so therefore we should start preparing for that now. That’s sort of my summary of maybe Stuart Russell’s book, Human Compatible, some of the work by Bostrom and other sort of famous existential risk people.

Jeremie Harris (06:18):
Superficially to me, that sounds bang on in terms of what keeps me up at night so to speak with this area. You’ve highlighted a lot of really interesting, I think areas where people who think in this way tend to be a little bit less aware of their considerations, they don’t weight as heavily as maybe they should. I’d love to get your thoughts on that aspect.

Melanie Mitchell (06:34):
I was particularly struck by the kinds of examples in the writings of some of these existential risk, I’m not sure what to call them, proponents warriors. They had examples like, here’s a scenario, we have a super-intelligent machine and we entrust it with solving the problem of climate change. And it decides that the best way to solve the problem of carbon emissions is to kill all the humans. So to me, that seems crazy. That’s just like saying that you could have a quote unquote super intelligent machine, but it wouldn’t have any common sense. It wouldn’t have any sense that human life is valuable to humans. And to make those kinds of things orthogonal, to me, misunderstand sort of what intelligence is all about.

Melanie Mitchell (07:31):
So that’s really… I feel that super intelligence itself, that notion is very ill defined, what people mean by that and whether it’s even possible to have something that is, what people have have said, much, much smarter than any human. Because intelligence is so complex, it’s so multi-dimensional. The dimensions of it are so entangles, or I believe, that I don’t think you can separate these things. So Boston for example, had what he called the orthogonality principle, which was that intelligence is orthogonal to objectives and values, and I just don’t believe that at all.

Jeremie Harris (08:20):
One of the things I was wondering about, with respect to the example you said with Stuart Russell and the climate change AI, that then destroys everything. The way I interpreted his claim was not necessarily that the mistake made by the AI would be that obvious, but more that systems are incredibly complicated. In general, one develop AIs that can solve problems in ways that we would not have thought of, and the risk is that those solutions are sort of dangerously creative, that we may kind of specify an objective for the AI to accomplish, but sort of like King Midas, everything he touches turns to gold, that’s what he wanted, but then he ends up turning his whole family to gold and so on.

Jeremie Harris (09:00):
You do end up with a situation where, for reasons that are subtle, not necessarily as obvious as to reduce CO2 emissions, well, let’s just kill everyone, but the AI comes up with a convoluted solution and that in the space of possible ways to solve a problem, the argument, I guess, is that the vast majority of points in that space lead actually to catastrophes as far as humans are concerned, when you optimize with respect to one metric. I’d love to get your thoughts on that because I’m sure you have some.

Melanie Mitchell (09:29):
I didn’t see that kind of subtlety in Russell’s arguments, but I think that is a good counter-argument to what I’m saying. And I think we do see that kind of thing, not only with machines, but with humans, that kind of, for example, the phenomenon of Bitcoin, it was created to decentralize currency and to enhance freedom and privacy and all of that. But there was a side effect that I don’t think people really understood until much later, which was the environmental impact of all this Bitcoin mining that goes on. And that could be contributing to an existential threat. And so the people who developed Bitcoin, obviously they’re super brilliant people, I don’t know if they foresaw that. So you could say, machine could do something like that. It could come up with a solution with these kinds of unintended negative side effects.

Melanie Mitchell (10:31):
But a machine can do that today, we see that today happening with things like the stock market flash crash and things where we give machines certain objectives that negative things can happen. My objection was more just to the idea of a super intelligence and what that means if… Sure we should have more insight into how machines are operating and how the objectives we give them can have different kinds of effects, but I don’t think that the argument is that there’s going to be some quote, unquote, super intelligent AI. And we say, oh, let’s just unleash this on the world and let it have the power to destroy everything. I just don’t see that as an immediate, near term threat to society. It’s more like using these machines that are non-intelligent, that are not intelligent at all and allowing them to have certain kinds of autonomy. That’s much more of a threat.

Jeremie Harris (11:44):
Right. And that’s really, I think, where there’s an interesting kind of differentiation between the two risks classes that you’ve been really good at kind of distinguishing between those two issues, which I think is also healthy because it’s so easy to confuse the risk from self-driving cars with the more existential risks that people talk about in this space. When it comes to the existential risk piece, the argument I think I’ve tended to see, it does, you’re right, come from a place of inherent, I don’t wanna say techno optimism, but at least the belief that super-intelligence is possible is definitely a core assumption. What are the things that cause you to think that super intelligence is, let’s say a highly improbable? Would that be a fair characterization?

Melanie Mitchell (12:24):
Well, I would say it’s more ill-defined. I don’t think we understand very much about intelligence scientifically. It’s a word that means many things. And there’s this sort of view in the computer science world that we can develop AI as kind of this brain in a VAT, if you will that’s sort of isolated from everything and getting some sensory input and then doing some output. And there’s no notion of kind of development, the way we have children learning and social interaction, being part of being part of a culture, growing up as part of a culture. And I don’t think you can easily separate those issues when you’re talking about intelligence. In some sense, all intelligence, at least the kind that we’re interested in having AI mimic is socially grounded. We humans the reasons for our big brains most likely is to deal with a complex social environment.

Melanie Mitchell (13:34):
And that is something that AI people think can kind of be separated from this sort of notion of more pure intelligence. And it should be said, people say, “Oh, we’re going to have these AI systems. They’re going to be super intelligent, but they’re not going to have any of our cognitive biases. They’re not going to have any of our faults, like getting tired and sleeping, and getting emotional or that kind of thing.” And the idea that there’s this separability among all these aspects of intelligence is something that I feel skeptical of. Maybe I’m wrong, but I feel we don’t really understand enough about intelligence to assume that there’s some kind of pure notion of intelligence that could be imbued or have machines learn somehow that is separated from all the other aspects.

Jeremie Harris (14:34):
It sounds you have a strong suspicion then that there’s some level of socialization required to get to general intelligence. Is that fair to say?

Melanie Mitchell (14:43):
That’s my suspicion.

Jeremie Harris (14:46):
Okay.

Melanie Mitchell (14:46):
Absolutely. And general intelligence again, it’s one of those terms that it’s hard to define because we don’t really know what we mean. Some people say there is no such thing. Some people argue there is no general intelligence. Humans have a specific kind of intelligence or they have certain kinds of skills and there’s no kind of notion of generality. Now, I don’t agree with that necessarily, but I think the term itself really needs to be fleshed out.

Jeremie Harris (15:12):
On the AI risks, sort of like, what should we do side of things, I guess then it becomes a game of probabilities to some degree. So there’s what probability do we assign to the thesis that intelligence needs to be socialized? What probability do we assign to the contrary thesis that says, “Yeah, actually we can build whatever with whatever?” And then how should we act in light of, I guess something like the expected returns? If there’s a 1% chance of like AI apocalypse then, well, presumably that’s worth acting on and the probabilities matter here. So for the sake of talking probabilities, what’s your gut feeling in terms of whether super intelligence is possible without socialization?

Melanie Mitchell (15:51):
It’s hard to speculate on super intelligence when I’m not even sure what it means. But I think Boston had a definition, which I can’t quote exactly, but it’s something like a system that’s able to do any task that a human can do and excel, do it faster, better, more accurately and so on. So I believe that without living in kind of the human social, cultural world without being kind of growing up in it, I would wager that it’s impossible to get to that kind of human intelligence.

Jeremie Harris (16:44):
Do you think that sociopaths would be a good counterpoint to this? Not to go straight to Hitler on this one, but Adolf Hitler, presumably very bright guy, unfortunately, but obviously had various kind of… He did not relate to human beings let’s say in the way that one should.

Melanie Mitchell (17:00):
I don’t think that’s really a valid point because, even a human sociopath, if you will, and I’m certainly not an expert on this, but it’s not they’re devoid of emotions. It’s not like they’re devoid of sociality. It’s not like they didn’t grow up and develop the way that other humans grew up and developed. There’s just some kind of deficit there. I don’t know exactly what it is, but they, people like that can function very well in society and can be fairly intelligent, but they can also cause great damage certainly. But I think that’s not a great analogy with a machine. Machine at least right now they have nothing, I mean, they’re just far away from any biological intelligence.

Jeremie Harris (17:58):
I guess I’m just wondering about the decoupling between the sort of socialization and almost like a caring. Is it caring for others? Would that be the main thing that you’d consider to be a critical part of this general intelligence?

Melanie Mitchell (18:10):
It’s more understanding, caring for others, but also understanding a kind of theory of mind of others and understanding their goals, understanding their motivations, understanding why, being able to predict them, to model them. That’s something that, I think a lot of people have argued that’s why we humans became so quote unquote intelligent, it’s in order to be able to successfully model and predict what other people in our group are doing, or why they’re doing what they’re doing. That’s something that’s lacking, certainly lacking in machines. And the thing about machines is that we want them to do things that require interaction with us, like driving or cleaning our houses, or a terrible example but very timely, fighting our wars. There has been a lot of talk about autonomous war fighters and all of that kind of thing. And fundamentally involves interacting with people and understanding other people. So that’s, I think something that people are trying to get AI systems to do now, but it’s very difficult and no one really knows how to do it well.

Jeremie Harris (19:35):
It definitely seems in a lot of these catastrophic scenarios where you have the kind of AI in the box, and then it sort of finds a way to break out, the cause her classic breakout scenario in AI safety, it does seem that those require some kind of theory of mind, as you say, because how are you going to manipulate? I think one of the classic examples, maybe Max Tegmark, might’ve used this one, but the janitor who walks by and then gets roped into helping the thing break out, you need a theory of mind for the janitor to convince them to do X, Y, and Z and so on. To the extent that an AI system could do that, if an AI system could let’s say predict the behavior of the, this is also fuzzy, but the median human being or something like that in some meaningful context, would that move the needle for you? If you saw that, you’d be, “Okay, I think that that can be done then divorced from the sort of socialization aspect.”

Melanie Mitchell (20:22):
Well, we already have systems that can do some kind of prediction of us, right?

Jeremie Harris (20:25):
Right.

Melanie Mitchell (20:26):
I mean, frighteningly like our social media platforms, fairly predict us all too well. But I think that they do it kind of in this associational way, they don’t have sort of rich models of human behavior. I don’t know if that would move the needle or not. I think that it’s essential for intelligence, it’s essential for being and acting intelligently in the world. So I think we have to be able to get our systems to do that. We have to get our self-driving cars to be able to predict what a human pedestrian is going to do or why they’re doing what they do. Would I find that more kind of risky? Well, I guess it’s, as the systems get more and more intelligent, we have to think about sort of what they can do autonomously and how much we can trust them.

Melanie Mitchell (21:25):
But I guess my point was that in order to get systems to be able to do that, I believe that they would have to, in some sense, grow up like our children, grow up in a culture, in a society. And in doing that, they would absorb their own values and their own objectives. I don’t think that you can have this sort of super intelligent yet strangely passive programmable system that the idea there, like Stuart Russell’s idea that we’re going to have a super intelligent system, but we have to have it absorb, learn our values. Well, there’s a way to do that, like with children, right? That’s called raising children. But raising children involves having an agent with the right kind of architecture and the right kind of learning environment and allowing them autonomy. They’re not passive, they’re active and they’re not just waiting to imitate us. They have their own autonomy.

Jeremie Harris (22:38):
And to the extent, I guess, that you have, let’s say super intelligence systems that are being raised, and I’m using a lot of air quotes here because I don’t know what I’m talking about. But anyway which is very much to your point. To the extent that we have systems like that, that are being raised in this way, I guess we do see sort of like misbehavior, again, Adolf Hitler had parents and Paul [inaudible 00:22:57] had parents, could you see an existential risk arising from a super intelligence system that meets all those criteria, but it’s just let’s say raised improperly, which might be another word for aligned improperly?

Melanie Mitchell (23:09):
It’s possible. I just see it as a very distant kind of risk. I certainly don’t think that this kind of quote unquote existential risk is impossible and it’s true with any technology, right? But I just think there’s a lot of near term risks that we should be focusing on as opposed to this very, what I think is a very longer-term risk that we don’t even quite know how to frame the problem exactly.

Jeremie Harris (23:44):
I think one thing might actually straddle the line, I do want to get into your thoughts on near-term risks that we really should be looking at. I suspect we’re going to be agreeing on an awful lot of stuff there. But the thing for me that straddles the line between near and far, and that I think has been a really interesting, it’s been interesting to see how people respond to it is GPT3, open AI’s large language model, obviously since then Google’s come out with a bigger one and so on. But this idea that scaling models seems to lead to a little bit more generality, a little bit more sort of flexibility, at least in terms of the use cases. I wouldn’t trust GPT3 to, I don’t know tell a doctor what steps to perform a surgery in, but you can imagine some [inaudible 00:24:23] GPT4 systems improve. To some people they’ve seen this as, okay, it looks like we’re on the way to generality just by scaling existing systems and maybe adding some spices and salt in there for extra flavor. But do you see that happening? Is that a convincing thing? Were you surprised by GPT3 in that way or not?

Melanie Mitchell (24:39):
I was not surprised by GPT3 in that way. I was surprised by it in some ways, because I think GPT3 is a language model. It doesn’t have any rich internal models of the world. It doesn’t know how the world works. You can see that by questioning it about the world and lots of people have published papers about that kind of thing, that it can just spout complete and total nonsense and things that are just factually wrong about the world, that no child would ever say. I was surprised at how good it is at generating seemingly coherent text. And that was… I think it’s doing it by statistical association, just like say a speech recognition program on your phone has learned a lot of statistical associations between sounds and words. And I’m surprised at how well those can do just based on that kind of approach.

Melanie Mitchell (25:44):
So I am always surprised by these kind of how this scaling these models to very large data sets and very large networks can improve them. I don’t believe that if you continue scaling GPT3 so that it memorizes all of human language, whatever, that it’s going to be a general intelligence. I don’t think that that’s sufficient. Maybe it can sound very convincing, but it’s not going to be able to do the kinds of things that we humans can do. It’s not going to be able to communicate in the kind of ways that we humans communicate. So it remains to be seen, but I don’t believe that’s going to happen.

Jeremie Harris (26:32):
This is an interesting question. What is the bar for us to say, okay, there’s actual reasoning going on here? Is it sufficient to show some examples where GBT3 seems to extrapolate beyond training set, seems to do things with like number addition with three digit numbers, stuff like that, even though it fails with more digits, there are some cases where it seems pretty clear that it hasn’t seen certain combinations of things and it still gets those right. But then there are, as you say, there are these glaring counterexamples where you’ll give it two facts in a row and then it’ll fail to kind of connect the two in different ways. So do you think that this issue of necessity versus sufficiency is an important thing that we should be talking about? Is it enough for these models to show some reasoning ability or do they need to be more robust before we declare them to be doing something interesting?

Melanie Mitchell (27:22):
I guess it depends what your goal is. If your goal is to build AI systems that will turn into products, then perhaps it’s sufficient for certain things. It’s maybe not as reliable as you’d like it to be, but it’s reliable a lot of the time, and if there’s a human in the loop, that’s great. But if your goal is to understand intelligence or to get to general intelligence, some kind, it’s not sufficient, it has to be more robust. This brings up a question of how do we tell, how do we know when something is generally intelligent? And most of my colleagues in AI absolutely hate the Turing test, but I’m actually more of a fan of it than most people I know.

Jeremie Harris (28:13):
Oh, cool. Okay. I’d love to hear your [inaudible 00:28:15].

Melanie Mitchell (28:17):
The way it’s been carried out so far, I think has been very flawed. It’s easy to fool people. It’s easy to fool judges. We’ve seen that for decades in AI, going back to Eliza, which fooled a lot of people.

Jeremie Harris (28:31):
And just for people who might not have dug into the trench, I think most people have a vague idea of how it works, but could you sort of lay out what the standard maybe Eliza version of the Turing test was?

Melanie Mitchell (28:42):
Right. So Turing himself wrote a paper about this proposing that you have a machine and a human that are sort of competing against each other to convince a judge that they are human. Okay. And they can only communicate using language and you can’t see the either because you don’t want to have what they look like influence you. So the judge, Turing proposed that, in his paper, he proposed that a five minute conversation might be revealing. Well, it turns out a five minute conversation can easily be created by a chat bot and not necessarily expert judge and fool the judge. Eliza was a simulated psychoanalyst that was developed the 1970s, I think, using just incredibly simple little templates and replies. It was one of the first chat bots. And actually people talked to it and they thought that it was understanding them. Because people are so prone to anthropomorphizing and to giving sort of agency to something they’re communicating with.

Melanie Mitchell (30:05):
So I don’t think that kind of Turing test tells us very much except about the gullibility of humans. But I do think that there are versions of the Turing tests that we might be able to construct that would really tell you a lot more now. There’s a famous long bet between Ray Kurzweil and Mitchell Kapore. So whether an AI system will be able to pass the Turing test by 2029, and they’re going to each bet $20,000 or something like that. And this was many years ago, they had this bet. But they set out this incredibly stringent Turing test with very expert judges and it lasts for hours and you can probe it in many different ways. And I reading that whole description of their Turing test, I thought, something passed that test, I would be very surprised, but also it would move the needle on my belief about sort of the system having some general intelligence.

Jeremie Harris (31:09):
The fact that you’re thinking in terms of what are some of the things that could convince me otherwise? I mean, this is something that I’d love to see more of in the space generally because so often it seems people’s aesthetic preferences are the dominating factor. I’m a person who tends to worry about the future. So of course I would have these concerns about existential risks from AGI. I’d be more receptive to that kind of argument. It’s just really cool to see it there. And the GPT3 angle, I mean, that seems to me to be an interesting tie into some of your more current concerns. So to the extent that we are worried about existential risk from AI, that does pull resources away from current concerns, and I think it’s important to highlight that trade off. Would you mind laying out maybe the most salient concerns, the most serious concerns you have about current use of the technology?

Melanie Mitchell (31:54):
I mean, GPT3 is a good starting point because this ability to create sort of fake media, fake text or images, videos, audio, whatever, it’s getting better and better at a very alarming rate. And we’ve seen these deep fakes improve enormously and it’s easy to fool people. So that can be a real… I feel like that’s going to be a really difficult issue for the spread of disinformation, propaganda, so on, very, very shortly, maybe it’s already starting to be. So that I think is an immediate risk. And these, the machines that create these fake media are not generally intelligent, they’re not super intelligence, they’re just are every day deep neural networks, what have you. So I think that’s a big problem. There’s also a big problem of racial and gender bias and other kinds of bias in these systems that a whole lot of people are trying to figure out what to do about now.

Melanie Mitchell (33:09):
And there’s also the problem of us sort of being too trusting and too optimistic, I think, about how reliable these systems are going to be. There was just recently a big report out from a committee that was looking into like autonomous weapons and it included Eric Schmidt, former CEO of Google and other technologists who really felt we are almost there, we’re almost at the point where we can deploy autonomous weapons and we’re going to figure out how to prove that they’re reliable and all of that. And I feel this kind of over techno optimism might lead us to some real problems. So those are more the immediate risks that I see. There’s other kinds of civil rights issues with privacy and surveillance and just a lot of things that are going on and dealing with international, figuring out international regulations on these technologies is a big problem.

Jeremie Harris (34:19):
Actually, this brings to mind another, I guess, variant almost on the existential risk argument that maybe is one that you’d probably find more convincing or more interesting, let’s say. And that is just the idea that as this technology improves, the destructive footprint of malicious actors starts to increase and our ability to do horrible things to ourselves as a species in much the same way that nuclear weapons just gave us an unfair way to kill each other, maybe this eventually culminates in something really disastrous. Is that something you consider plausible?

Melanie Mitchell (34:48):
Absolutely. I think there’s the vulnerability to malicious actors. We’ve seen that time and time again, like cybersecurity and with deploying all these AI systems and all of our devices in our cars, our electrical grid, everything it’s just opening us up to a lot of vulnerability.

Jeremie Harris (35:08):
And how do you think about the [inaudible 00:35:10]? You alluded to regulation and, I think, I’m a big fan of the idea that we are going to have to regulate somehow the space. I guess, it’s a complex question of how to do that, but it’s in the context as well of this awful game theoretic dilemma too, where you have the United States, you have China, various other global powers, all vying for dominance, all developing this technology. One of them can of course, step back and say, “We will not develop autonomous weapons,” but then of course why other people? This is probably a question best post like an AI policy walk, but just because you’ve done so much thinking in this space, do you have any kind of intuitions about where there might be some traction to be heard on addressing that issue?

Melanie Mitchell (35:49):
That’s a really difficult issue. And as you say, it’s not my area of expertise, but I think that like nuclear weapons, you’re going to have to use… We’re going to have to have treaties, and diplomacy, and international pressure. To me, it’s kind of analogous to the situation right now in like bio engineering, genetic engineering, which is another source of possible existential risk, if you will. And we’re struggling with how to regulate that, how to regulate it internationally using the UN and other international bodies. I think the same thing’s going to happen with AI technology, but it’s a very difficult problem.

Jeremie Harris (36:42):
And is there anything that technologists can do, researchers for example, focusing their research efforts in specific directions that you think might help? Can AI be part of the solution as well as the problem?

Melanie Mitchell (36:53):
Yes. I think there’s a lot of things people could do, ways to make AI more robust, more trustworthy and that’s multi-dimensional, but more transparent. One of the problems is that these AI systems are black boxes. It’s hard to figure out what they’re doing, how to sort of certify them as you were. Somebody proposed having an FDA for algorithms. How would you certify that an algorithm is safe the way you do with a new drug or something like that? It’s going to involve a lot of breakthroughs that will be needed to do that kind of thing. But this is something that computer science has been working on for a very long time, how to sort of do verification of algorithms and kind of prove that they’re doing what they’re supposed to be doing. There’s just a lot of research that can be done in this area.

Jeremie Harris (38:01):
In your book too, I think you wrote about GDPR and sort of how the right to explanation plays into all this as well. I’d love to get your take on that, because I think some people see the GDPR thing is overbearing, but other people see it as necessary. And then it seems you’ve got a pretty nuanced take about what an explanation is and sort of inviting a lot of philosophical questions around that.

Melanie Mitchell (38:22):
Well, it’s a difficult question because so GDPR is the European union’s law, part of which says, “If an algorithm is going to affect your life, decide you’re going to get a loan, or if you can get approved for housing, or if you’re going to go to prison or whatever, it needs to be able to explain its reasoning.” So if you have a deep neural network with a billion parameters, it’s hard to say what is the reasoning of this thing besides giving, here’s all the weights. That’s all I got. Right?

Jeremie Harris (38:57):
[inaudible 00:38:57].

Melanie Mitchell (38:58):
Right. That doesn’t help us. That’s not how we understand things. And the explanation has to be tailored to a particular person’s understanding. So what does an explanation, that’s a philosophical question, right? And I don’t think it’s a real morass for the legal profession, if they’re going to have a lawsuit and say, “This algorithm has to explain itself.” Well, what exactly does that mean? Let’s get some expert witness philosophers in here to talk about the nature of explanation and [inaudible 00:39:30] and all. It’s going to be a mess. I don’t know how that’s all going to play out. It’s really interesting. But they talk about AI taking away jobs, but I think it’s going to create a lot of new jobs because it’s going to create a whole new area of law, of philosophy, of a kind of moral philosophy of people assessing risks, that kind of thing.

Jeremie Harris (40:02):
I usually hear that argument made in the context of I guess software eats the world and don’t worry about it, we’ll have more developers, but inevitably of course, developer time is better leverage than the tasks that it automates away, which is why it’s economically profitable. But in this case, I guess we really are discovering that we have to do philosophy on a deadline, we have to really start ramping up our efforts. Are you optimistic, generally speaking, I mean, you’re so steeped in the technical side of this, but on the human side, do you think that we’re going to be able to adapt our society in time for a lot of these changes?

Melanie Mitchell (40:36):
No. I mean, it’s already shown that we can’t.

Jeremie Harris (40:39):
Well, I was hoping for something more optimistic.

Melanie Mitchell (40:42):
No, no, we have not adapted our society. And we’ve run into things like this whole epidemic of disinformation on social media, which has had extreme real-world consequences. And we haven’t been able to adapt. And even the tech companies are trying to adapt. They’re trying to regulate themselves. It turns out to be a lot more difficult than anybody imagined. And technology alone is not going to solve the problem, we have to have I think policy and regulation. But that means educating a whole lot of people who are not tech people into being able to understand what’s going on in these fields. I just gave a guest lecture in a class at a law school of students who are thinking about technology and the law and realizing that this is just going to be an exploding area in the near future. So they have to know something about it.

Jeremie Harris (41:45):
Every time I talk to anybody who’s in AI policy, we’ve done a couple of those podcasts and it just seems this idea of the, I think they call it the pacing problem. Just the idea that the policy and regulation just struggles to keep pace with technology, especially when it rides these exponential curves. It just seems people are wondering about in many cases, a pretty fundamental re drawings of the line in terms of how systems work for policymaking, just to make them more responsive. Do you think something like that is going to have to happen? Are we going to have to iterate almost at a kind of startup pace on our laws, which sounds really scary to me as I say it, but is that something that you see happening?

Melanie Mitchell (42:22):
I don’t know, because it seems laws and regulations go so very, very slowly. You watch these hearings in Congress about social media to algorithm bias and so on. And the lawmakers are just scratching their heads because they don’t understand any of it. They don’t have any idea how it works.

Jeremie Harris (42:41):
Very comforting.

Melanie Mitchell (42:44):
So I can’t imagine that the laws and the regulations can keep up with the technology.

Jeremie Harris (42:53):
So that being the case, it seems to put a lot of responsibility on the shoulders of researchers, since those seem to be the people best placed to actually do something about this. As a researcher, how has this thinking kind of effected your focus, the sorts of things you’ve tried to draw attention to?

Melanie Mitchell (43:10):
Well, for one thing, it makes me realize that my education as a technical person was almost completely lacking in anything about the social impacts of technology. And that I think perhaps more quickly than the laws and regulations is going to catch up, it’s more the technology education is going to try and put more emphasis on social impacts. And I see that happening in computer science departments all over this country, at least, that ethics and social impacts are becoming part of the curriculum. People are starting to really focus on that. I don’t know how helpful that’s going to be. I think we’ll see. But it’s really important at least for people to be aware of these issues.

Jeremie Harris (44:04):
I guess, as you said with law, education moves slowly, kind of on generational time in some cases. Hopefully we’ll be picking that up. And hopefully our AIs can help us learn faster. Boy, this becomes a whole thing. Well, Melanie, thanks so much. I really appreciate it. I really appreciate all your insights. I do want to recommend your book by the way, especially for people who are listening to the podcast, who’ve been longtime listeners. We do talk a lot about the kind of AI risk side of things. If you want other perspective, I really recommend going out and getting the book. It is called Artificial Intelligence: A Guide For Thinking Humans. And I guess, Melanie it’s available on your website.

Melanie Mitchell (44:40):
They can find out where to where to get it. Yes, absolutely.

Jeremie Harris (44:43):
Okay. So we’ll provide a link to that in the blog post, they’ll come with a podcast as well. And Melanie, thanks so much again.

Melanie Mitchell (44:48):
Thank you. It’s been a great conversation.