Jacob Coxon

It's one of the paradoxes of the current situation that some of the darkest warnings about the dangers posed by AI are coming from within the very companies and laboratories that are leading the development of the technology.

The current scare, if you want to call it that, was probably ignited by a social media post by Jacob Coxon, a young researcher at Anthropic who announced his resignation while noting that the people building AI earnestly believe that it could kill us all by the end of the decade.

Rather than deny Coxon's warning, Evan Hubinger, a senior employee at Anthropic working on AI safety, replied, Jacob is correct.

We really do believe AI could kill us all by the end of the decade.

I, I'm curious in how Chinese thinking on some of those specific threats has developed and evolved.

Let's focus on, you know, the kinds of existential concerns that I think came to at least wider appreciation in the wake of some recent incidents, and especially the, uh, resignation of Anthropic researcher Jacob Coxon, who warned of, you know, relatively high likelihood of All of humanity being killed by AI in the, the, the next decade or so.

Kyle, when Chinese policymakers and leading Chinese thinkers here read that kind of warning and see the debate that it precipitates in the United States, what do they think of it? Does it resonate with them? Do they see similar concerns? Does it seem hysterical and overblown? Does it validate their model? I mean, there's any, uh, probably a range of views, but how, how would you break down the, the Chinese version of this debate and their reaction to what's happening here?

I think the Chinese reaction is they're sort of scratching their heads about the existential risk issue.

You literally can't open a news app without seeing it.

And this current panic cycle, it really kicked off with this guy, Jacob Coxon.

He's an ex-employee of the AI company Anthropic.

Right.

Like just grabbing a coffee might delete later.

Humans might go extinct.

It's almost surreal how casually these catastrophic ideas are floated into the public discourse by these tech guys.

But here is the thing.

But it seems like it's come to the forefront in just the last couple of weeks.

The major spark here was on September 8th, a guy named Jacob Coxon, he's a 27-year-old pre-training researcher who had worked at OpenAI and Anthropic for a couple of years, he quit Anthropic and made a post that said that basically both of these companies are not acting responsibly.

They are racing to try and reach superintelligence, and in the process are, quote, "gambling with our lives." Now, this went viral to say the least.

That post alone got 90 million views in 24 hours, and it's sparked a much wider debate.

Let's just say I'm skeptical, Catherine.

So we had this one employee at Anthropic named Jacob Coxon who quit and posted something to Twitter X that's now been viewed something like 170 million times, which tells you that it hit a note.

And the note was basically he was screaming, the machines are about to rise up and kill us all.

And people who may not be deep in this could take that.

But I'm not sitting back pretending that there are no risks.

Just a few weeks ago, we saw the anthropic researcher Jacob Coxon announce his resignation from the company, saying that AI could, quote, kill us all by the end of the decade.

We're no longer talking about hypothetical dangers raised by critics, random critics of technology.

The companies building the systems are putting some of those dangers in their own investor disclosures.

Welcome back.

This week, artificial intelligence researcher Jacob Coxon set off alarm bells across the country when he resigned from AI giant Anthropic, writing, quote, the people building AI earnestly believe that it could kill us all by the end of the decade.

This is not a marketing stunt.

If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible.

Owls also are symbolic.

So they're like, okay, he's smart because he's got horn-rimmed tortoiseshell glasses.

Jacob Coxon, welcome to Meet the Press.

Nice to be on.

We've all heard it by now, but why not continue to fearmonger here on the show? Former researchers at leading AI companies saying themselves that there's a greater than 10% chance AI will kill all of us by the end of the decade.

Here's an ex-anthropic researcher, he's one of these guys, Jacob Coxon on CNN a couple of weeks ago.

I think that if you extrapolate into the future the level of capabilities of these AIs with the same independent volition, they could cause extreme havoc.

For example, hacking critical infrastructure, building extinction-level bioweapons.