ABOUT DR. KENTARO TOYAMA:
Kentaro Toyama is W. K. Kellogg Professor of Community Information at the University of Michigan School of Information and a fellow of the Dalai Lama Center for Ethics and Transformative Values at MIT. Previously, Kentaro taught at Ashesi University in Ghana and worked at Microsoft Research, where he co-founded its India lab and did research in AI, human-computer interaction, and applications of digital technology for community development. He is the author of "Geek Heresy: Rescuing Social Change from the Cult of Technology."
RESOURCES:
Geek Heresy: Rescuing Social Change from the Cult of Technology
NPR: "Anthopic researcher resigns amid AI safety concerns"
TRANSCRIPTION:
Caroline MacGregor: You're listening to 89.1 WEMU. I'm Caroline MacGregor, and my guest today is Kentaro Toyama. And he is a computer scientist who studies the relationship between technology and global development. He's the W.K. Kellogg Professor at the University of Michigan's School of Information. And he's also the author of "Geek Heresy: Rescuing Social Change from the Cult of Technology." Today, we are talking about artificial intelligence researchers' concerns that AI could be the apocalypse that ends humanity. And we are hearing more and more about this in the news headlines each day. Kentaro, welcome to the program!
Dr. Kentaro Toyama: Thanks, Caroline, for having me!
Caroline MacGregor: So, to start off, I just want to say, before an anthropic researcher resigned and declared human extinction imminent last week, tech leaders and scientists had already been sounding the alarm about a super-intelligent AI ending humanity for more than a decade. What are your thoughts about the legitimacy of this concern?
Dr. Kentaro Toyama: I think that an AI doomsday is definitely possible--if not right now, then in the near future. I think the real question now is timeframe and scale, meaning how soon does a catastrophe of a particular scale become possible. So, I think of it in scientific notation by the number of zeros in the people affected. Like today, my guess is we're probably at already level four or five, a disaster that could harm tens or hundreds of thousands of people. I don't think we're yet at level nine. That would be billions of people, but AI is moving fast enough that level nine could be years, maybe even months away.
Caroline MacGregor: Good gosh! I mean, this is not very encouraging news. What specific pathways could realistically lead from advanced AI to human extinction?
Dr. Kentaro Toyama: Yeah. So, extinction, I think, is a pretty strong possibility. And I think that's fairly far out for other reasons. But in terms of even smaller scale disasters, any critical infrastructure that's connected to the internet is vulnerable to AI attacks. And those targets are the kind that we all see in science fiction, like air traffic control systems, power grids, water supplies, police departments, telecom networks, and so on. If you take any of them down, it's chaos for thousands of people, at the least, and some of it will be fatal. For example, a city without power for a week means patients on medical devices dying, water and sanitation collapsing, credit cards and Apple Pay not working, food and refrigerators rotting. You'd basically eventually have civil unrest, and I haven't even mentioned autonomous weapons or killer robots.
Caroline MacGregor: Is there a chance that AI could cause the end of humanity accidentally without having a particular goal of harming humans?
Dr. Kentaro Toyama: So, I'm skeptical that, like, the total end of humanity, meaning all human beings dead is an easy possibility. I mean, one possibly, I guess, is a nuclear winter, possibly an AI system could start a war that ends up in a nuclear war. But for example, human beings have been pretty good about keeping the most dangerous weapons under fairly strict security controls. So, for example, I don't see that, even with today's AI, it would be so easy to launch a nuclear weapon. Those systems are quite secure and involve lots of human beings. They're air gapped from the internet, things like that. But again, a big disaster that might still harm millions, I think, is very possible.
Caroline MacGregor: Now, OpenAI CEO Sam Altman sent out a message this past Tuesday, saying the world should trust that AI scientists are going to do the right thing because it is the right thing and because he says they feel the magnitude of this. This was stated during a conference this week in San Francisco. What do you think of his comments?
Dr. Kentaro Toyama: Well, first of all, I would say Sam Altman is a flip-flopper on this issue. He sometimes is begging the government for industry regulation, and sometimes, he says things like you just quoted. But I think the real issue with what he's saying in that particular instance is that we are now at a point where we have a technology that, in some ways, is already smarter than us, though not in every way, and that just means that even with the best of intentions, there's no guarantee that we can truly control it. And so, I worry that this is a kind of hubris, right? Like, you know there are a lot of commentators who are saying things like, well, we can't expect chimpanzees to control human beings. Why are we expecting human beings to control an AI system that is much, much smarter than we are? I think that's the real crux, and that's not a problem that can be solved, no matter how good the intentions are.
Caroline MacGregor: Just talking to you, it's hard not to be deeply troubled by all of this. Now, this has, of course, come into the headlines more recently following the departure of a Google deep mind researcher, an anthropic researcher. He warned that artificial intelligence could "kill us all," obviously adding to a lot of other people sounding the alarm from inside the industry about a technology that the U,S. President is currently dismissing as a doomsday hoax. How would you respond to the President of the US calling this a doomsday hoax?
Dr. Kentaro Toyama: It's definitely not a hoax. I do think "kill us all" is a pretty strong claim. And I think human beings are shockingly resilient. So, I think killing us all down to the last person is very difficult, in fact. But I do think that major catastrophes are possible because of AI, and we do have to take these claims seriously. You know, you were referring to Kozan, I think, who left Anthropic because he was so concerned. He was saying that every leader at Anthropic is very concerned about these kinds of, if not doomsday, then catastrophic scenarios. And I think those are legitimate concerns.
Caroline MacGregor: The highly respected British preeminent professor, Stephen Hawking, he also expressed concerns about this before he died. He was concerned that artificial intelligence could end humanity.
Dr. Kentaro Toyama: You know, I don't mean to say that it's an impossibility, I think it's possible. It's possible that it would be triggered by AI in some form. The main reason why I don't think total extinction is so readily going to happen is that I suspect we will see many smaller disasters happen before extinction, and if any of disasters is large enough, I'm hopeful that human beings will wake up and possibly even do what they did in the story arc of the science fiction novel "Dune," in which there was a revolution among human beings to get rid of all thinking machines. And I think something like that is more likely than that we'll suddenly end up in a nuclear war with a zero warning whatsoever.
Caroline MacGregor: I want to ask you. How do researchers define and measure AI alignment currently?
Dr. Kentaro Toyama: AI alignment is simply the idea that AI should operate according to values that we would like to invest in it, such as protect human life or ensure that you're not intentionally harming a person if you can help it, that kind of thing. But alignment is something that we would like in each other as well. And one of the biggest challenges is that it's not at all clear what set of values, or who is the set of values, should be invested in AI. At the same time, I think, technically, we're having a very difficult time as a community of AI scientists actually getting these systems to do exactly what we want them to do. Even people who have used AI casually know that they're notoriously difficult to have followed instructions, especially very detailed instructions. And today's AI is just not reliable with respect to that.
Caroline MacGregor: Do you believe governments should impose limits on the development of frontier AI?
Dr. Kentaro Toyama: I do think that some regulation is essential. I think the most important thing is to pass laws that make any human being who is responsible for developing, releasing, or prompting an AI that causes harm to be personally and probably criminally liable for that harm in proportion to the harm that's caused. I think that would very quickly stop the kind of carelessness that we now routinely expect from Silicon Valley. They're basically treating us all as guinea pigs as they develop these models. If they all knew, everybody involved in developing systems, that they could be held prudently liable if, for example, somebody who uses an AI system commits suicide, I think that they would be much, much, much more careful. But having said that, even there, it's unclear that that would be enough because, again, we're trying to control something that could be already smarter than us and will certainly be smarter than us in the near future.
Caroline MacGregor: How can countries coordinate to reduce catastrophic AI risks?
Dr. Kentaro Toyama: So, some people make this analogy between AI and nuclear weapons. And I actually think that's apt, although there are many ways in which AI is more difficult to regulate. But with nuclear weapons, the world went through a period where they worked very hard to politically limit the number of countries that had nuclear weapons. And the surprising thing with the Nuclear Non-Proliferation Treaty is that, because of it, today, we only have 10 or 11 countries that have nuclear weapons out of 200. That's a surprising and impressive political feat that I think we could duplicate again. I think China is ready to come to the negotiating table with the United States to put some regulation around AI development, but, of course, that means we have to do the right kind of diplomacy.
Caroline MacGregor: That was my next question. I mean, politically speaking, the U.S. currently is not very inclined to work very closely with China. With our current administration here in the U.S., how likely do you think it is that this issue will be taken seriously?
Dr. Kentaro Toyama: Yeah. Unfortunately, I think our current administration is extremely gung-ho about AI. They've already tried to suppress AI regulation at the state level, and it seems like, more than anything, they just want more and more AI development. I worry that that's the wrong approach and also that it doesn't help us with negotiating with other countries a kind of a worldwide AI regulation.
Caroline MacGregor: Are you personally very concerned about all of this?
Dr. Kentaro Toyama: I am very concerned about AI. I kind of hopeful that some of the worst kind of AI disasters will, in fact, be held in check by these AI companies who I think are beginning to see just how dangerous the fire it is that they're playing with. But I do think there are many, many other challenges that AI will pose to the entire world. And, yeah, those are scary. Widespread joblessness is pretty scary, even if you just think in terms of the potential civil unrest that that might lead to. The possibility of accelerating climate change is scary. All of these things are frightening. And, yes, we desperately need leaders who are mindful of those possibilities and doing what they can to address them.
Caroline MacGregor: All right. And in closing, what safeguards in your mind should be mandatory before deploying highly autonomous AI?
Dr. Kentaro Toyama: Yeah. There's a lot of talk of various technical safeguards. I actually really think it's the regulatory ones that are the most important, which is, again, to make sure that the human beings who are behind the development of AI and the systems and who might use it to cause harm intentionally or otherwise should be legally held liable for the harm that ensues. That's really the main thing. We talk about autonomous AI and so on. But they're still machines. The human beings created them and are deploying them, we're the ones who should be ultimately held responsible. At least in the United States, people seem very conscious of the various harms that AI can cause, whether it's joblessness or on the climate change, and I think that's good. We as citizens really need to take political action to constrain the development of these technologies.
Caroline MacGregor: I've been speaking with Kentaro Toyama. He's a computer scientist, and we've been talking about artificial intelligence and researchers' concerns that AI could be the apocalypse that ends humanity. It's a disturbing conversation, Kentaro, but thank you very much for joining us today.
Dr. Kentaro Toyama: Thank you! And good luck to us all!
Caroline MacGregor: This is 89.1 WEMU Ypsilanti.
Non-commercial, fact based reporting is made possible by your financial support. Make your donation to WEMU today to keep your community NPR station thriving.
Like 89.1 WEMU on Facebook and follow us on X (Twitter)
Contact WEMU News at 734.487.3363 or email us at studio@wemu.org
