Global catastrophic risk

And literally, I think around the same time that the book was put to bed, this opportunity came along and it was exactly what I was looking for.

For those familiar, even for those not familiar with the bulletin, it originally focused on nuclear weapons, nuclear issues, But its mandate is existential risk, anything that could pose a risk to humanity, ending humanity.

So today those include AI, other forms of disruptive technology, bio threats, climate.

And for me, these were exactly the issues that I have focused on in my career.

What does technology mean for us for national security and as a society? So it was just kind of perfect timing.

And it's a great publication.

I know you've only been there a short time, but have you noticed any differences from working at a nonprofit focused on existential risk versus being in a large newsroom?

A place like The Wall Street Journal brings tremendous resources to bear on an issue when Russia invaded Ukraine, when Hamas launched attacks on Israel, and then the outbreak of the war in Gaza.

Seth Baum is co-founder and executive director of Global Catastrophic Risk Institute.

He is also a research affiliate of the Center for the Study of Existential Risk at the University of Cambridge.

He holds several advanced degrees and completed a postdoc with the Center for Research on Environmental Decisions at Columbia University.

And he was also a fellow of the Emerging Leaders in Biosecurity Program of the Center for Health Security at Johns Hopkins.

And what we do as an organization is work across the full space of these risks to characterize the risk and help to develop solutions for reducing the risk.

That's wonderful.

I know there are a number of other similar organizations that are out there now that are devoted to the study of existential risk.

Some of these that did not exist even 10 years ago.

What are the most important problems I can work on? He loves thinking big.

And one of the first things that he thinks of is existential risk and making humanity multi-planetary.

from his perspective if humanity only lives on earth as we do now then we are always just one asteroid away from extinction so this is a big concern for elon and so he starts looking around and realizes man why is it that we started exploring space in the 50s we went to the moon a few times and then we haven't been back to the moon since 1972. we should be going further into space exploring more not retreating from space we should be going to mars why isn't anyone trying to do this and so yes it's about existential risk but it's also about something more inspirational reading now from the isaacson biography quote part of his motivation came from his heritage in a family of adventurers and his decision as a teenager to move to a country that had bred into its essence the spirit of pioneers The United States is literally a distillation of the human spirit of exploration.

This is a land of adventurers.

To have a base on Mars would be incredibly difficult, and people will probably die along the way, just as happened in the settling of the United States.

But it will be incredibly inspiring, and we must have inspiring things in the world.

Okay, so it's about existential risk, but it's also about doing something exciting and daring and stoking that spirit of pioneering and exploration.

Now, initially, his mindset is what everyone's mindset was before SpaceX, which is that space exploration is the domain of government entities of programs like NASA.

Just going on, I'd like to focus a little bit more on the risk.

There's been much talk of the existential risk of AI.

Proposals include, I think the House of Lords recently suggested a kill switch.

If it all goes horribly wrong, that was rejected by government.

So that's what AI has done to the speed of development in probably only the last 18 months to two years, if that.

So I think it's really interesting how AI has sped up the landscape.

But actually, if you ask the people who do cybersecurity, who are on the front line, They're concerned about these things, but they're not saying that it's an existential risk.

And actually, I think a lot of people are looking in the wrong place.

No, no, no.

What I'm saying is that I think that that, not I think, the number of people internally at all of the large corporations, like the AI companies, who thought that AI was an existential risk- didn't change in private polls it didn't change what changed was all of those people's comfort in being like oh yeah no i also think that that's happening and so it's not that like they all read that and then they were like oh man that's not what i'm pointing

to that's

not what i'm

Exactly.

Yet with artificial intelligence, billions of dollars are flowing into the exact mechanisms creating this 35% existential risk, and society as a whole is largely cheering it on.

Which forces us to ask you, the listener, why the blind spot? Why are we as a global hyper-connected society completely fine with rolling the dice on human extinction?

Just so we can have language models draft our quarterly earnings reports?

If things are bad enough, there may well be an attempt to shut down AI or shut down further AI development or both.

That is all true even if, incorrectly, you think the truly terrible scenarios won't happen, and we do not face substantial existential risk.

Those scenarios, and the future in general, should be gamed out too.

This is an excellent use of time.

Quote, Dave Caston.

One smaller update I had from conversations over the past two months, which snapped into focus for me at the curve, is that models are getting destructive faster than they're getting good at planning and scheming.

So we should expect a bunch of dumb catastrophic stuff to happen well before we get to existential risk territory.

Andrew Retek.

Hmm.

Some kind of existential risk.

That's kind of the business that we're in.

And more to the point, though, that is a no AI guarantee on the novel bio.

They said, "Well, obviously this could wipe out all life on Earth"

[laughs]

... uh, which they specifically said, "Catastrophic or existential risks to humanity."

Mm-hmm.