Sep 24, 2026 · 1 hr 18 min · 18 segments
Jerusalem Demsas and Matt Yglesias debate whether artificial intelligence poses an existential risk to human civilization, or if that idea is just a dangerous distraction from real, near-term risks…
Matthew YglesiasHost
Jerusalem DemsasHost
Well, I'm concerned that advanced artificial intelligence will kill all of us.
No, so I think the reason is that there is a chance, I'm not saying a high chance, but a real chance that artificial intelligence has been becoming more and more capable lately and that it will soon be capable enough that it can automate the work of developing next generation artificial intelligence, which means that we may see 10 times more progress in the next five years than we've seen in the last in a way that really kind of changes the world and leaves us with a world in which powerful, autonomous, agentic artificial intelligence is embedded all throughout everything that everybody does everywhere.
I mean, just even in our house, right? Like I would love to have a robot that folds my laundry and tidies up every day after I leave the house.
And we find ourselves in a world where AI has sort of total control over everything that is happening.
And then at some point— finds that it is inconvenient to be maintaining this large human population that is consuming a lot of resources and getting in the way of other objectives that may or may not exist out there.
And I think this is important not because I think that's necessarily the most likely thing that is going to happen, but because if you think about what are ways that human civilization even could be wiped out, there's not that many like that that would be hard to do um one of the main ones is that the earth could be hit by a really really really large asteroid right that also probably won't i i don't like lay in terror at night of that happening but like physically that would wipe out everybody and space objects do collide and a you know low-key take of mine is that like we should invest more money in monitoring like far earth astronomical objects so that we could avoid this Really powerful AI also has that characteristic that it could be so capable and so universally embedded that it's not just like something really bad happens, but something completely unrecoverable happens.
And to me, at least, because there's some people out there, there was a book recently called If Anyone Builds It, Everyone Dies.
Everyone could die and like almost nothing else that I think of, things that people worry about all the time and that usually are legitimately bad just don't have that same level of downside risk because the universality of like really hyper-powerful AI will be unique.
It will be everywhere and it will be in control of everything because it will be smarter and better than us and it will be like our – It's like how we humans have caused the extinction of lots and lots and lots of lesser animals.
Like I don't think people – All kinds of like random lizards and things get driven to extinction just by the nature of human activity.
And if we create like a new entity that is more capable than us, there's just inherently going to be this risk of being shoved aside.
In the AI discourse, I've long been considered more of an optimistic person because my concerns are more in the realm of I think millions of people could die and there could be mass blackouts and someone could release biological weapons that kill millions of people.
But I've been very skeptical of people who talk a lot or think a lot about extinction.
And I just want to be clear, that's an optimistic kind of person in this discourse, which I think speaks to how scared I think many people are about the outcomes that are possible here.
But I mean, I think that one thing that has made me more bearish on a lot of these extinction things, I hear that you're saying you're not like one of these people who is like, you know, really certain this is going to happen.
Like if AI systems are built, if recursive self-improvement, which is the process that you just described happens.
around how ai can just build the next generation of ai by itself would inevitably lead to extinction like you're not saying that but part of why i feel really skeptical about this is because i think there's so much in between an ai system becoming hyper intelligent and it getting power there are a lot of people out there who are incredibly brilliant people and like they could even know right now like what it would take to shut down the entire american electricity grid But like they actually can't do it because power is more than just like intelligence.
And this is a point that the folks who wrote AI is normal technology, which I think is a really useful, useful piece, make a lot.
Well, I'm concerned that advanced artificial intelligence will kill all of us.
No, so I think the reason is that there is a chance, I'm not saying a high chance, but a real chance that artificial intelligence has been becoming more and more capable lately and that it will soon be capable enough that it can automate the work of developing next generation artificial intelligence, which means that we may see 10 times more progress in the next five years than we've seen in the last in a way that really kind of changes the world and leaves us with a world in which powerful, autonomous, agentic artificial intelligence is embedded all throughout everything that everybody does everywhere.
I mean, just even in our house, right? Like I would love to have a robot that folds my laundry and tidies up every day after I leave the house.
And we find ourselves in a world where AI has sort of total control over everything that is happening.
And then at some point— finds that it is inconvenient to be maintaining this large human population that is consuming a lot of resources and getting in the way of other objectives that may or may not exist out there.
And I think this is important not because I think that's necessarily the most likely thing that is going to happen, but because if you think about what are ways that human civilization even could be wiped out, there's not that many like that that would be hard to do um one of the main ones is that the earth could be hit by a really really really large asteroid right that also probably won't i i don't like lay in terror at night of that happening but like physically that would wipe out everybody and space objects do collide and a you know low-key take of mine is that like we should invest more money in monitoring like far earth astronomical objects so that we could avoid this Really powerful AI also has that characteristic that it could be so capable and so universally embedded that it's not just like something really bad happens, but something completely unrecoverable happens.
And to me, at least, because there's some people out there, there was a book recently called If Anyone Builds It, Everyone Dies.
Everyone could die and like almost nothing else that I think of, things that people worry about all the time and that usually are legitimately bad just don't have that same level of downside risk because the universality of like really hyper-powerful AI will be unique.
It will be everywhere and it will be in control of everything because it will be smarter and better than us and it will be like our – It's like how we humans have caused the extinction of lots and lots and lots of lesser animals.
Like I don't think people – All kinds of like random lizards and things get driven to extinction just by the nature of human activity.
And if we create like a new entity that is more capable than us, there's just inherently going to be this risk of being shoved aside.
In the AI discourse, I've long been considered more of an optimistic person because my concerns are more in the realm of I think millions of people could die and there could be mass blackouts and someone could release biological weapons that kill millions of people.
But I've been very skeptical of people who talk a lot or think a lot about extinction.
And I just want to be clear, that's an optimistic kind of person in this discourse, which I think speaks to how scared I think many people are about the outcomes that are possible here.
But I mean, I think that one thing that has made me more bearish on a lot of these extinction things, I hear that you're saying you're not like one of these people who is like, you know, really certain this is going to happen.
Like if AI systems are built, if recursive self-improvement, which is the process that you just described happens.
around how ai can just build the next generation of ai by itself would inevitably lead to extinction like you're not saying that but part of why i feel really skeptical about this is because i think there's so much in between an ai system becoming hyper intelligent and it getting power there are a lot of people out there who are incredibly brilliant people and like they could even know right now like what it would take to shut down the entire american electricity grid But like they actually can't do it because power is more than just like intelligence.
And this is a point that the folks who wrote AI is normal technology, which I think is a really useful, useful piece, make a lot.
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.