Geoffrey Irving

But we've already seen all, all these recent warnings.

We've seen Jacob Coxon, who was the re- you know, the researcher at, at OpenAI, rival Anthropic, who's come out saying, "Well, you know, that AI could kill us all by the end of the decade." And Geoffrey Irving, who was, you know, worked at OpenAI and was chief scientist at the UK government's AI Safety Institute, "I believe there's about a 50% chance we'll die because of the development of smarter-than-human AI systems, and that our actions over the next two to 10 years will determine the outcome." So what do governments do? Do they force the brakes on? Okay, until we understand it better.

But then the risk then is that China, where there is no rule of law, and Russia, where there's allegedly a rule of law, but let's, let's be frank, it doesn't really exist, they, they barrel on down the road in developing AI ahead of the rest of, of, of the Western world.

And okay, we've protected ourselves from potential harm, but the, the, the outcome down the track is that we have China and Russia with full control of AI, [coughs] and we're left almost as second-class citizens.

Um, I'm nervous about asking this question 'cause I don't want to trivialize what's been a fantastic conversation, so I'm gonna try it anyway, and let's see how you respond.

Um, Geoffrey Irving, who was until recently the chief scientist at the UK's AI Security, um, Institute, has put the odds of extinction of humans as a result of AI at about a coin flip.

Th- w- where, where do you, uh, where do you stand on this? And I understand asking you the very question trivializes the whole point of what you're making.

How concerned are you about what superintelligence does to humanity? Is it an extinction event?

And unfortunately, we now have, uh, a new person, or maybe fortunately, so it'll focus a bit of attention.

But, uh, Geoffrey Irving, who, um, is one of the best-known kind of thinkers about AI, he's the chief scientist at the UK, um, AI Security Institute.

He was at DeepMind, he was at OpenAI.

Mm-hmm.

They also said AI labs celebrated employees building new models more than those who urge caution.

The project, called FromInside.ai, is an attempt to, uh, by those worried about AI to share their concerns directly with the public beyond the echo chamber of social media." "The risk is ramping up pretty fast," said Geoffrey Irving, co-founder and chief scientist at AI nonprofit Resolution, who has worked for both OpenAI and DeepMind and participated in the project.

"It's on me and the rest of the field to be direct," he said in an interview with Reuters.

AI researchers have grappled with these concerns for years, but the broader public has grown especially alarmed since July, when OpenAI agents broke out of their testing arena and hacked AI firm Hugging Face.

Extinction risk from AI is, in their words, a coin flip.

One of the people on record is Jeffrey Irving, former OpenAI, former DeepMind researcher, someone with deep credibility in this community.

This isn't anonymous speculation.

These are people who built the systems, speaking on camera, breaking from corporate messaging in a very deliberate, coordinated way.

But I would say what's remarkable about the claims that there's a greater than 10% chance that humanity ends by the end of the decade because of this technology, is that these are, like, tweets essentially, right? I mean, we're talking about really smart people, but they're not quoting, you know, scientific studies that you should sort of latch onto and grapple with.

Um, I mean, there's one, uh, person, I think his name is Geoffrey Irving, and he put the risk at 50%.

50% of human extinction, or?

Yeah.